[BioMart Users] Problem building 3NF virtual mart

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

[BioMart Users] Problem building 3NF virtual mart

Thomas TRIPLET
Hello,

I'm trying to build a mart (v0.8rc6) from a normalized (3NF) PostgreSQL database. So far, it looks really great, except for one issue I have. When I look at the source database schema (using MartConfigurator), it is just fine, so the database seems properly imported. However, the corresponding target database really messes things up as it tries to merge everything together. Here is a simple example to illustrate the problem:

sourceDB.png  targetDB.png
  

The target database is clearly problematic, and not usable as such. Is there a way to force BioMart to keep the source schema untouched, or at least, control how it is processed ? ALl relations in the source database are defined as [1:M] so I'm not sure what's wrong here.

Also, it is my understanding that a database must exist in the form of a database in order to be imported (whether it is actually accessed using RDBMS/URL/registry), or can it also be defined as raw datafiles (e.g GFF files) along with a parser. Could you please confirm this?

Thanks a lot for your help.

Thomas

--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6





_______________________________________________
Users mailing list
[hidden email]
https://lists.biomart.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: [BioMart Users] Problem building 3NF virtual mart

Syed Haider-2

Thomas,

Converting your source database into a mart is not mandatory, its only
recommended for performance reasons. Try using the source schema &
create a dataset straight out of your source database (without
materialising to mart) and the subsequent querying and web interfaces
should work just fine (at least in theory).

W.r.t your second question, Martconfigurator is not able to read GFF or
other text file formats. The input to Martconfigurator should either be
an existing biomart webserver end point or a database.

HTH,
Syed


On 04/10/2011 22:03, Thomas TRIPLET wrote:

> Hello,
>
> I'm trying to build a mart (v0.8rc6) from a normalized (3NF) PostgreSQL database. So far, it looks really great, except for one issue I have. When I look at the source database schema (using MartConfigurator), it is just fine, so the database seems properly imported. However, the corresponding target database really messes things up as it tries to merge everything together. Here is a simple example to illustrate the problem:
>
> [cid:ii_132d0b1d058ca382]  [cid:ii_132d0b234e4777ae]
>
>
> The target database is clearly problematic, and not usable as such. Is there a way to force BioMart to keep the source schema untouched, or at least, control how it is processed ? ALl relations in the source database are defined as [1:M] so I'm not sure what's wrong here.
>
> Also, it is my understanding that a database must exist in the form of a database in order to be imported (whether it is actually accessed using RDBMS/URL/registry), or can it also be defined as raw datafiles (e.g GFF files) along with a parser. Could you please confirm this?
>
> Thanks a lot for your help.
>
> Thomas
>
> --
> Thomas Triplet, Ph.D.
> http://www.thomastriplet.net
>
> Centre for Structural and Functional Genomics
> Concordia University
> 7141 West Sherbrooke St
> Montreal QC H4B 1R6
>
>
>
>
_______________________________________________
Users mailing list
[hidden email]
https://lists.biomart.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: [BioMart Users] Problem building 3NF virtual mart

Thomas TRIPLET
Syed,
Thanks for your reply. Sorry I wasn't clear enough, I am trying to use the data source as it is, not materializing it as a mart. But when I add the data source as a Postgres RDBMS, MartConfigurator automatically uses the source schema to build the target schema. Is it possible to disable this feature and force MartConfigurator to use the same schema for the target database as for the source database?
Thanks
Thomas


--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






On Wed, Oct 5, 2011 at 9:07 AM, Syed Haider <[hidden email]> wrote:

Thomas,

Converting your source database into a mart is not mandatory, its only recommended for performance reasons. Try using the source schema & create a dataset straight out of your source database (without materialising to mart) and the subsequent querying and web interfaces should work just fine (at least in theory).

W.r.t your second question, Martconfigurator is not able to read GFF or other text file formats. The input to Martconfigurator should either be an existing biomart webserver end point or a database.

HTH,
Syed



On 04/10/2011 22:03, Thomas TRIPLET wrote:
Hello,

I'm trying to build a mart (v0.8rc6) from a normalized (3NF) PostgreSQL database. So far, it looks really great, except for one issue I have. When I look at the source database schema (using MartConfigurator), it is just fine, so the database seems properly imported. However, the corresponding target database really messes things up as it tries to merge everything together. Here is a simple example to illustrate the problem:

[cid:ii_132d0b1d058ca382]  [cid:ii_132d0b234e4777ae]



The target database is clearly problematic, and not usable as such. Is there a way to force BioMart to keep the source schema untouched, or at least, control how it is processed ? ALl relations in the source database are defined as [1:M] so I'm not sure what's wrong here.

Also, it is my understanding that a database must exist in the form of a database in order to be imported (whether it is actually accessed using RDBMS/URL/registry), or can it also be defined as raw datafiles (e.g GFF files) along with a parser. Could you please confirm this?

Thanks a lot for your help.

Thomas

--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






_______________________________________________
Users mailing list
[hidden email]
https://lists.biomart.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: [BioMart Users] Problem building 3NF virtual mart

Arek Kasprzyk
Hi Thomas,
you lost me here. Why would you want to use the same schema for source and target? Could you elaborate on this please?

a

On Wed, Oct 5, 2011 at 11:24 AM, Thomas TRIPLET <[hidden email]> wrote:
Syed,
Thanks for your reply. Sorry I wasn't clear enough, I am trying to use the data source as it is, not materializing it as a mart. But when I add the data source as a Postgres RDBMS, MartConfigurator automatically uses the source schema to build the target schema. Is it possible to disable this feature and force MartConfigurator to use the same schema for the target database as for the source database?
Thanks
Thomas


--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






On Wed, Oct 5, 2011 at 9:07 AM, Syed Haider <[hidden email]> wrote:

Thomas,

Converting your source database into a mart is not mandatory, its only recommended for performance reasons. Try using the source schema & create a dataset straight out of your source database (without materialising to mart) and the subsequent querying and web interfaces should work just fine (at least in theory).

W.r.t your second question, Martconfigurator is not able to read GFF or other text file formats. The input to Martconfigurator should either be an existing biomart webserver end point or a database.

HTH,
Syed



On 04/10/2011 22:03, Thomas TRIPLET wrote:
Hello,

I'm trying to build a mart (v0.8rc6) from a normalized (3NF) PostgreSQL database. So far, it looks really great, except for one issue I have. When I look at the source database schema (using MartConfigurator), it is just fine, so the database seems properly imported. However, the corresponding target database really messes things up as it tries to merge everything together. Here is a simple example to illustrate the problem:

[cid:ii_132d0b1d058ca382]  [cid:ii_132d0b234e4777ae]



The target database is clearly problematic, and not usable as such. Is there a way to force BioMart to keep the source schema untouched, or at least, control how it is processed ? ALl relations in the source database are defined as [1:M] so I'm not sure what's wrong here.

Also, it is my understanding that a database must exist in the form of a database in order to be imported (whether it is actually accessed using RDBMS/URL/registry), or can it also be defined as raw datafiles (e.g GFF files) along with a parser. Could you please confirm this?

Thanks a lot for your help.

Thomas

--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






_______________________________________________
Users mailing list
[hidden email]
https://lists.biomart.org/mailman/listinfo/users



_______________________________________________
Users mailing list
[hidden email]
https://lists.biomart.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: [BioMart Users] Problem building 3NF virtual mart

Thomas TRIPLET
Hi Arek,

This is because my source schema is normalized. When added as a RDBMS data source in BioMart, it is processed to build the target schema. The problem is that the target schema that is generated is denormalized and unusable. Here are 2 screenshots to illustrate the issue:
I hope this clarifies my problem.

Thanks
Thomas

--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






On Wed, Oct 5, 2011 at 11:39 AM, Arek Kasprzyk <[hidden email]> wrote:
Hi Thomas,
you lost me here. Why would you want to use the same schema for source and target? Could you elaborate on this please?

a

On Wed, Oct 5, 2011 at 11:24 AM, Thomas TRIPLET <[hidden email]> wrote:
Syed,
Thanks for your reply. Sorry I wasn't clear enough, I am trying to use the data source as it is, not materializing it as a mart. But when I add the data source as a Postgres RDBMS, MartConfigurator automatically uses the source schema to build the target schema. Is it possible to disable this feature and force MartConfigurator to use the same schema for the target database as for the source database?
Thanks
Thomas


--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






On Wed, Oct 5, 2011 at 9:07 AM, Syed Haider <[hidden email]> wrote:

Thomas,

Converting your source database into a mart is not mandatory, its only recommended for performance reasons. Try using the source schema & create a dataset straight out of your source database (without materialising to mart) and the subsequent querying and web interfaces should work just fine (at least in theory).

W.r.t your second question, Martconfigurator is not able to read GFF or other text file formats. The input to Martconfigurator should either be an existing biomart webserver end point or a database.

HTH,
Syed



On 04/10/2011 22:03, Thomas TRIPLET wrote:
Hello,

I'm trying to build a mart (v0.8rc6) from a normalized (3NF) PostgreSQL database. So far, it looks really great, except for one issue I have. When I look at the source database schema (using MartConfigurator), it is just fine, so the database seems properly imported. However, the corresponding target database really messes things up as it tries to merge everything together. Here is a simple example to illustrate the problem:

[cid:ii_132d0b1d058ca382]  [cid:ii_132d0b234e4777ae]



The target database is clearly problematic, and not usable as such. Is there a way to force BioMart to keep the source schema untouched, or at least, control how it is processed ? ALl relations in the source database are defined as [1:M] so I'm not sure what's wrong here.

Also, it is my understanding that a database must exist in the form of a database in order to be imported (whether it is actually accessed using RDBMS/URL/registry), or can it also be defined as raw datafiles (e.g GFF files) along with a parser. Could you please confirm this?

Thanks a lot for your help.

Thomas

--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






_______________________________________________
Users mailing list
[hidden email]
https://lists.biomart.org/mailman/listinfo/users




_______________________________________________
Users mailing list
[hidden email]
https://lists.biomart.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: [BioMart Users] Problem building 3NF virtual mart

Arek Kasprzyk
Hi Thomas,
What happens if you launch the server and query it? If you do not materilize, the QueryCompiler will ignore the target schema and translate SQL against the source instead.

Could you try querying it to see if you are getting the results back?

a


On Wed, Oct 5, 2011 at 11:51 AM, Thomas TRIPLET <[hidden email]> wrote:
Hi Arek,

This is because my source schema is normalized. When added as a RDBMS data source in BioMart, it is processed to build the target schema. The problem is that the target schema that is generated is denormalized and unusable. Here are 2 screenshots to illustrate the issue:
I hope this clarifies my problem.

Thanks
Thomas

--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






On Wed, Oct 5, 2011 at 11:39 AM, Arek Kasprzyk <[hidden email]> wrote:
Hi Thomas,
you lost me here. Why would you want to use the same schema for source and target? Could you elaborate on this please?

a

On Wed, Oct 5, 2011 at 11:24 AM, Thomas TRIPLET <[hidden email]> wrote:
Syed,
Thanks for your reply. Sorry I wasn't clear enough, I am trying to use the data source as it is, not materializing it as a mart. But when I add the data source as a Postgres RDBMS, MartConfigurator automatically uses the source schema to build the target schema. Is it possible to disable this feature and force MartConfigurator to use the same schema for the target database as for the source database?
Thanks
Thomas


--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






On Wed, Oct 5, 2011 at 9:07 AM, Syed Haider <[hidden email]> wrote:

Thomas,

Converting your source database into a mart is not mandatory, its only recommended for performance reasons. Try using the source schema & create a dataset straight out of your source database (without materialising to mart) and the subsequent querying and web interfaces should work just fine (at least in theory).

W.r.t your second question, Martconfigurator is not able to read GFF or other text file formats. The input to Martconfigurator should either be an existing biomart webserver end point or a database.

HTH,
Syed



On 04/10/2011 22:03, Thomas TRIPLET wrote:
Hello,

I'm trying to build a mart (v0.8rc6) from a normalized (3NF) PostgreSQL database. So far, it looks really great, except for one issue I have. When I look at the source database schema (using MartConfigurator), it is just fine, so the database seems properly imported. However, the corresponding target database really messes things up as it tries to merge everything together. Here is a simple example to illustrate the problem:

[cid:ii_132d0b1d058ca382]  [cid:ii_132d0b234e4777ae]



The target database is clearly problematic, and not usable as such. Is there a way to force BioMart to keep the source schema untouched, or at least, control how it is processed ? ALl relations in the source database are defined as [1:M] so I'm not sure what's wrong here.

Also, it is my understanding that a database must exist in the form of a database in order to be imported (whether it is actually accessed using RDBMS/URL/registry), or can it also be defined as raw datafiles (e.g GFF files) along with a parser. Could you please confirm this?

Thanks a lot for your help.

Thomas

--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






_______________________________________________
Users mailing list
[hidden email]
https://lists.biomart.org/mailman/listinfo/users





_______________________________________________
Users mailing list
[hidden email]
https://lists.biomart.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: [BioMart Users] Problem building 3NF virtual mart

Thomas TRIPLET
Hi Arek,

I tried to add only the tables from my previous example and started the servers without materializing them.
It seems that the QueryCompiler does not ignore the target schema (cf screenshot: http://ttriplet.fg.concordia.ca/literatureTrial.png).

As far as I can tell with such a simple example, querying works in the sense that data is retrieved, but the forms and filters that must be populated for querying are not intuitive nor practical.

Also, it might be a related fact/error, when I import the 4 tables altogether, 2 data sources are created (one called literature, one person). I noticed this usually happens when the FKs in the source schema are not properly defined, but this is not the case here.

Also, the log gave me the following when I imported the tables:

org.biomart.common.exceptions.ValidationException: You can have at most one each of 1:M and M:1 subclass relations from the same table, but no more.
  at org.biomart.objects.objects.Relation.setSubclassRelation(Relation.java:447)
  at org.biomart.configurator.controller.MartController.continueSubclassing(MartController.java:1562)
  at org.biomart.configurator.controller.MartController.suggestMarts(MartController.java:1367)
  at org.biomart.configurator.controller.MartController.requestCreateMartsFromSource(MartController.java:1599)
  at org.biomart.configurator.controller.ObjectController.initMarts(ObjectController.java:98)
  at org.biomart.configurator.view.component.container.SourceGroupPanel$1.construct(SourceGroupPanel.java:85)
  at org.biomart.common.view.gui.SwingWorker$2.run(SwingWorker.java:128)
  at java.lang.Thread.run(Thread.java:679)

So it looks like the problem is that one table has two 1:M relations (probably the table used to define the M:N relation between articles and authors).
Thanks a lot for your help.

--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






On Wed, Oct 5, 2011 at 12:31 PM, Arek Kasprzyk <[hidden email]> wrote:
Hi Thomas,
What happens if you launch the server and query it? If you do not materilize, the QueryCompiler will ignore the target schema and translate SQL against the source instead.

Could you try querying it to see if you are getting the results back?

a



On Wed, Oct 5, 2011 at 11:51 AM, Thomas TRIPLET <[hidden email]> wrote:
Hi Arek,

This is because my source schema is normalized. When added as a RDBMS data source in BioMart, it is processed to build the target schema. The problem is that the target schema that is generated is denormalized and unusable. Here are 2 screenshots to illustrate the issue:
I hope this clarifies my problem.

Thanks
Thomas

--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






On Wed, Oct 5, 2011 at 11:39 AM, Arek Kasprzyk <[hidden email]> wrote:
Hi Thomas,
you lost me here. Why would you want to use the same schema for source and target? Could you elaborate on this please?

a

On Wed, Oct 5, 2011 at 11:24 AM, Thomas TRIPLET <[hidden email]> wrote:
Syed,
Thanks for your reply. Sorry I wasn't clear enough, I am trying to use the data source as it is, not materializing it as a mart. But when I add the data source as a Postgres RDBMS, MartConfigurator automatically uses the source schema to build the target schema. Is it possible to disable this feature and force MartConfigurator to use the same schema for the target database as for the source database?
Thanks
Thomas


--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






On Wed, Oct 5, 2011 at 9:07 AM, Syed Haider <[hidden email]> wrote:

Thomas,

Converting your source database into a mart is not mandatory, its only recommended for performance reasons. Try using the source schema & create a dataset straight out of your source database (without materialising to mart) and the subsequent querying and web interfaces should work just fine (at least in theory).

W.r.t your second question, Martconfigurator is not able to read GFF or other text file formats. The input to Martconfigurator should either be an existing biomart webserver end point or a database.

HTH,
Syed



On 04/10/2011 22:03, Thomas TRIPLET wrote:
Hello,

I'm trying to build a mart (v0.8rc6) from a normalized (3NF) PostgreSQL database. So far, it looks really great, except for one issue I have. When I look at the source database schema (using MartConfigurator), it is just fine, so the database seems properly imported. However, the corresponding target database really messes things up as it tries to merge everything together. Here is a simple example to illustrate the problem:

[cid:ii_132d0b1d058ca382]  [cid:ii_132d0b234e4777ae]



The target database is clearly problematic, and not usable as such. Is there a way to force BioMart to keep the source schema untouched, or at least, control how it is processed ? ALl relations in the source database are defined as [1:M] so I'm not sure what's wrong here.

Also, it is my understanding that a database must exist in the form of a database in order to be imported (whether it is actually accessed using RDBMS/URL/registry), or can it also be defined as raw datafiles (e.g GFF files) along with a parser. Could you please confirm this?

Thanks a lot for your help.

Thomas

--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






_______________________________________________
Users mailing list
[hidden email]
https://lists.biomart.org/mailman/listinfo/users






_______________________________________________
Users mailing list
[hidden email]
https://lists.biomart.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: [BioMart Users] Problem building 3NF virtual mart

Arek Kasprzyk
Hi Thomas,
sorry, i did not make myself clear. The GUI gets configured as if it was talking to the target schema. However, the QueryComplier queries the source not target. The target simply does not physically exist at this point unless you materilize it. If you want to make sure that the source is being queried you can switch the logging on to see the actual SQL that is being compiled. As far as the GUI is concerned you can use MConfigurator to give it any shape of form that you require. It does not have to look like an image of the target schema. This is just a default behaviour to get you started

a

On Wed, Oct 5, 2011 at 1:25 PM, Thomas TRIPLET <[hidden email]> wrote:
Hi Arek,

I tried to add only the tables from my previous example and started the servers without materializing them.
It seems that the QueryCompiler does not ignore the target schema (cf screenshot: http://ttriplet.fg.concordia.ca/literatureTrial.png).

As far as I can tell with such a simple example, querying works in the sense that data is retrieved, but the forms and filters that must be populated for querying are not intuitive nor practical.

Also, it might be a related fact/error, when I import the 4 tables altogether, 2 data sources are created (one called literature, one person). I noticed this usually happens when the FKs in the source schema are not properly defined, but this is not the case here.

Also, the log gave me the following when I imported the tables:

org.biomart.common.exceptions.ValidationException: You can have at most one each of 1:M and M:1 subclass relations from the same table, but no more.
  at org.biomart.objects.objects.Relation.setSubclassRelation(Relation.java:447)
  at org.biomart.configurator.controller.MartController.continueSubclassing(MartController.java:1562)
  at org.biomart.configurator.controller.MartController.suggestMarts(MartController.java:1367)
  at org.biomart.configurator.controller.MartController.requestCreateMartsFromSource(MartController.java:1599)
  at org.biomart.configurator.controller.ObjectController.initMarts(ObjectController.java:98)
  at org.biomart.configurator.view.component.container.SourceGroupPanel$1.construct(SourceGroupPanel.java:85)
  at org.biomart.common.view.gui.SwingWorker$2.run(SwingWorker.java:128)
  at java.lang.Thread.run(Thread.java:679)

So it looks like the problem is that one table has two 1:M relations (probably the table used to define the M:N relation between articles and authors).
Thanks a lot for your help.

--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






On Wed, Oct 5, 2011 at 12:31 PM, Arek Kasprzyk <[hidden email]> wrote:
Hi Thomas,
What happens if you launch the server and query it? If you do not materilize, the QueryCompiler will ignore the target schema and translate SQL against the source instead.

Could you try querying it to see if you are getting the results back?

a



On Wed, Oct 5, 2011 at 11:51 AM, Thomas TRIPLET <[hidden email]> wrote:
Hi Arek,

This is because my source schema is normalized. When added as a RDBMS data source in BioMart, it is processed to build the target schema. The problem is that the target schema that is generated is denormalized and unusable. Here are 2 screenshots to illustrate the issue:
I hope this clarifies my problem.

Thanks
Thomas

--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






On Wed, Oct 5, 2011 at 11:39 AM, Arek Kasprzyk <[hidden email]> wrote:
Hi Thomas,
you lost me here. Why would you want to use the same schema for source and target? Could you elaborate on this please?

a

On Wed, Oct 5, 2011 at 11:24 AM, Thomas TRIPLET <[hidden email]> wrote:
Syed,
Thanks for your reply. Sorry I wasn't clear enough, I am trying to use the data source as it is, not materializing it as a mart. But when I add the data source as a Postgres RDBMS, MartConfigurator automatically uses the source schema to build the target schema. Is it possible to disable this feature and force MartConfigurator to use the same schema for the target database as for the source database?
Thanks
Thomas


--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






On Wed, Oct 5, 2011 at 9:07 AM, Syed Haider <[hidden email]> wrote:

Thomas,

Converting your source database into a mart is not mandatory, its only recommended for performance reasons. Try using the source schema & create a dataset straight out of your source database (without materialising to mart) and the subsequent querying and web interfaces should work just fine (at least in theory).

W.r.t your second question, Martconfigurator is not able to read GFF or other text file formats. The input to Martconfigurator should either be an existing biomart webserver end point or a database.

HTH,
Syed



On 04/10/2011 22:03, Thomas TRIPLET wrote:
Hello,

I'm trying to build a mart (v0.8rc6) from a normalized (3NF) PostgreSQL database. So far, it looks really great, except for one issue I have. When I look at the source database schema (using MartConfigurator), it is just fine, so the database seems properly imported. However, the corresponding target database really messes things up as it tries to merge everything together. Here is a simple example to illustrate the problem:

[cid:ii_132d0b1d058ca382]  [cid:ii_132d0b234e4777ae]



The target database is clearly problematic, and not usable as such. Is there a way to force BioMart to keep the source schema untouched, or at least, control how it is processed ? ALl relations in the source database are defined as [1:M] so I'm not sure what's wrong here.

Also, it is my understanding that a database must exist in the form of a database in order to be imported (whether it is actually accessed using RDBMS/URL/registry), or can it also be defined as raw datafiles (e.g GFF files) along with a parser. Could you please confirm this?

Thanks a lot for your help.

Thomas

--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






_______________________________________________
Users mailing list
[hidden email]
https://lists.biomart.org/mailman/listinfo/users







_______________________________________________
Users mailing list
[hidden email]
https://lists.biomart.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: [BioMart Users] Problem building 3NF virtual mart

Thomas TRIPLET
Hi Arek,

Thanks for your message, that's definitely  very helpful. Still got a question about the target database as I am trying to think in the longer term. At some point, as more data comes in, I will probably have to materialize the virtual mart as recommended in the documentation, so that queries are executed in a reasonable amount of time. I'm not there yet, but at that point, it will be the target database that will be used (since it is the one that is materialized). So my initial question regarding the target schema still holds for longer term perspectives. If MartConfigurator can't control how the target schema is generated, is it possible to manually edit the underlying registry file so that the source and target schema match?

Also, this might sound a bit stupid, but how do I activate the logs for the web UI. I've set DEBUG to ON in dist/biomart.properties, but the dist/logs folder doesn't get populated (I've restarted the servers).

Thanks!
Thomas

--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






On Wed, Oct 5, 2011 at 1:45 PM, Arek Kasprzyk <[hidden email]> wrote:
Hi Thomas,
sorry, i did not make myself clear. The GUI gets configured as if it was talking to the target schema. However, the QueryComplier queries the source not target. The target simply does not physically exist at this point unless you materilize it. If you want to make sure that the source is being queried you can switch the logging on to see the actual SQL that is being compiled. As far as the GUI is concerned you can use MConfigurator to give it any shape of form that you require. It does not have to look like an image of the target schema. This is just a default behaviour to get you started

a


On Wed, Oct 5, 2011 at 1:25 PM, Thomas TRIPLET <[hidden email]> wrote:
Hi Arek,

I tried to add only the tables from my previous example and started the servers without materializing them.
It seems that the QueryCompiler does not ignore the target schema (cf screenshot: http://ttriplet.fg.concordia.ca/literatureTrial.png).

As far as I can tell with such a simple example, querying works in the sense that data is retrieved, but the forms and filters that must be populated for querying are not intuitive nor practical.

Also, it might be a related fact/error, when I import the 4 tables altogether, 2 data sources are created (one called literature, one person). I noticed this usually happens when the FKs in the source schema are not properly defined, but this is not the case here.

Also, the log gave me the following when I imported the tables:

org.biomart.common.exceptions.ValidationException: You can have at most one each of 1:M and M:1 subclass relations from the same table, but no more.
  at org.biomart.objects.objects.Relation.setSubclassRelation(Relation.java:447)
  at org.biomart.configurator.controller.MartController.continueSubclassing(MartController.java:1562)
  at org.biomart.configurator.controller.MartController.suggestMarts(MartController.java:1367)
  at org.biomart.configurator.controller.MartController.requestCreateMartsFromSource(MartController.java:1599)
  at org.biomart.configurator.controller.ObjectController.initMarts(ObjectController.java:98)
  at org.biomart.configurator.view.component.container.SourceGroupPanel$1.construct(SourceGroupPanel.java:85)
  at org.biomart.common.view.gui.SwingWorker$2.run(SwingWorker.java:128)
  at java.lang.Thread.run(Thread.java:679)

So it looks like the problem is that one table has two 1:M relations (probably the table used to define the M:N relation between articles and authors).
Thanks a lot for your help.

--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






On Wed, Oct 5, 2011 at 12:31 PM, Arek Kasprzyk <[hidden email]> wrote:
Hi Thomas,
What happens if you launch the server and query it? If you do not materilize, the QueryCompiler will ignore the target schema and translate SQL against the source instead.

Could you try querying it to see if you are getting the results back?

a



On Wed, Oct 5, 2011 at 11:51 AM, Thomas TRIPLET <[hidden email]> wrote:
Hi Arek,

This is because my source schema is normalized. When added as a RDBMS data source in BioMart, it is processed to build the target schema. The problem is that the target schema that is generated is denormalized and unusable. Here are 2 screenshots to illustrate the issue:
I hope this clarifies my problem.

Thanks
Thomas

--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






On Wed, Oct 5, 2011 at 11:39 AM, Arek Kasprzyk <[hidden email]> wrote:
Hi Thomas,
you lost me here. Why would you want to use the same schema for source and target? Could you elaborate on this please?

a

On Wed, Oct 5, 2011 at 11:24 AM, Thomas TRIPLET <[hidden email]> wrote:
Syed,
Thanks for your reply. Sorry I wasn't clear enough, I am trying to use the data source as it is, not materializing it as a mart. But when I add the data source as a Postgres RDBMS, MartConfigurator automatically uses the source schema to build the target schema. Is it possible to disable this feature and force MartConfigurator to use the same schema for the target database as for the source database?
Thanks
Thomas


--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






On Wed, Oct 5, 2011 at 9:07 AM, Syed Haider <[hidden email]> wrote:

Thomas,

Converting your source database into a mart is not mandatory, its only recommended for performance reasons. Try using the source schema & create a dataset straight out of your source database (without materialising to mart) and the subsequent querying and web interfaces should work just fine (at least in theory).

W.r.t your second question, Martconfigurator is not able to read GFF or other text file formats. The input to Martconfigurator should either be an existing biomart webserver end point or a database.

HTH,
Syed



On 04/10/2011 22:03, Thomas TRIPLET wrote:
Hello,

I'm trying to build a mart (v0.8rc6) from a normalized (3NF) PostgreSQL database. So far, it looks really great, except for one issue I have. When I look at the source database schema (using MartConfigurator), it is just fine, so the database seems properly imported. However, the corresponding target database really messes things up as it tries to merge everything together. Here is a simple example to illustrate the problem:

[cid:ii_132d0b1d058ca382]  [cid:ii_132d0b234e4777ae]



The target database is clearly problematic, and not usable as such. Is there a way to force BioMart to keep the source schema untouched, or at least, control how it is processed ? ALl relations in the source database are defined as [1:M] so I'm not sure what's wrong here.

Also, it is my understanding that a database must exist in the form of a database in order to be imported (whether it is actually accessed using RDBMS/URL/registry), or can it also be defined as raw datafiles (e.g GFF files) along with a parser. Could you please confirm this?

Thanks a lot for your help.

Thomas

--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






_______________________________________________
Users mailing list
[hidden email]
https://lists.biomart.org/mailman/listinfo/users








_______________________________________________
Users mailing list
[hidden email]
https://lists.biomart.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: [BioMart Users] Problem building 3NF virtual mart

Arek Kasprzyk
Hi Thomas,
I am not entirely sure what you mean by 'MartConfigurator can't control how the target schema is generated'. MConfigurator executes the 3NF schema transformation algorithm that i designed while still at EBI and that has been described in the recent paper:

http://database.oxfordjournals.org/content/2011/bar038.full.pdf+html

The target schema will be a query optimized version of the source schema. The target schema complies with the reverse star spec so it will always be different from the source. However, the schemas and SQL against them are functionally equivalent so they will return the same data.

Please let me know if you need more information
a


On Wed, Oct 5, 2011 at 4:05 PM, Thomas TRIPLET <[hidden email]> wrote:
Hi Arek,

Thanks for your message, that's definitely  very helpful. Still got a question about the target database as I am trying to think in the longer term. At some point, as more data comes in, I will probably have to materialize the virtual mart as recommended in the documentation, so that queries are executed in a reasonable amount of time. I'm not there yet, but at that point, it will be the target database that will be used (since it is the one that is materialized). So my initial question regarding the target schema still holds for longer term perspectives. If MartConfigurator can't control how the target schema is generated, is it possible to manually edit the underlying registry file so that the source and target schema match?

Also, this might sound a bit stupid, but how do I activate the logs for the web UI. I've set DEBUG to ON in dist/biomart.properties, but the dist/logs folder doesn't get populated (I've restarted the servers).

Thanks!
Thomas

--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






On Wed, Oct 5, 2011 at 1:45 PM, Arek Kasprzyk <[hidden email]> wrote:
Hi Thomas,
sorry, i did not make myself clear. The GUI gets configured as if it was talking to the target schema. However, the QueryComplier queries the source not target. The target simply does not physically exist at this point unless you materilize it. If you want to make sure that the source is being queried you can switch the logging on to see the actual SQL that is being compiled. As far as the GUI is concerned you can use MConfigurator to give it any shape of form that you require. It does not have to look like an image of the target schema. This is just a default behaviour to get you started

a


On Wed, Oct 5, 2011 at 1:25 PM, Thomas TRIPLET <[hidden email]> wrote:
Hi Arek,

I tried to add only the tables from my previous example and started the servers without materializing them.
It seems that the QueryCompiler does not ignore the target schema (cf screenshot: http://ttriplet.fg.concordia.ca/literatureTrial.png).

As far as I can tell with such a simple example, querying works in the sense that data is retrieved, but the forms and filters that must be populated for querying are not intuitive nor practical.

Also, it might be a related fact/error, when I import the 4 tables altogether, 2 data sources are created (one called literature, one person). I noticed this usually happens when the FKs in the source schema are not properly defined, but this is not the case here.

Also, the log gave me the following when I imported the tables:

org.biomart.common.exceptions.ValidationException: You can have at most one each of 1:M and M:1 subclass relations from the same table, but no more.
  at org.biomart.objects.objects.Relation.setSubclassRelation(Relation.java:447)
  at org.biomart.configurator.controller.MartController.continueSubclassing(MartController.java:1562)
  at org.biomart.configurator.controller.MartController.suggestMarts(MartController.java:1367)
  at org.biomart.configurator.controller.MartController.requestCreateMartsFromSource(MartController.java:1599)
  at org.biomart.configurator.controller.ObjectController.initMarts(ObjectController.java:98)
  at org.biomart.configurator.view.component.container.SourceGroupPanel$1.construct(SourceGroupPanel.java:85)
  at org.biomart.common.view.gui.SwingWorker$2.run(SwingWorker.java:128)
  at java.lang.Thread.run(Thread.java:679)

So it looks like the problem is that one table has two 1:M relations (probably the table used to define the M:N relation between articles and authors).
Thanks a lot for your help.

--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






On Wed, Oct 5, 2011 at 12:31 PM, Arek Kasprzyk <[hidden email]> wrote:
Hi Thomas,
What happens if you launch the server and query it? If you do not materilize, the QueryCompiler will ignore the target schema and translate SQL against the source instead.

Could you try querying it to see if you are getting the results back?

a



On Wed, Oct 5, 2011 at 11:51 AM, Thomas TRIPLET <[hidden email]> wrote:
Hi Arek,

This is because my source schema is normalized. When added as a RDBMS data source in BioMart, it is processed to build the target schema. The problem is that the target schema that is generated is denormalized and unusable. Here are 2 screenshots to illustrate the issue:
I hope this clarifies my problem.

Thanks
Thomas

--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






On Wed, Oct 5, 2011 at 11:39 AM, Arek Kasprzyk <[hidden email]> wrote:
Hi Thomas,
you lost me here. Why would you want to use the same schema for source and target? Could you elaborate on this please?

a

On Wed, Oct 5, 2011 at 11:24 AM, Thomas TRIPLET <[hidden email]> wrote:
Syed,
Thanks for your reply. Sorry I wasn't clear enough, I am trying to use the data source as it is, not materializing it as a mart. But when I add the data source as a Postgres RDBMS, MartConfigurator automatically uses the source schema to build the target schema. Is it possible to disable this feature and force MartConfigurator to use the same schema for the target database as for the source database?
Thanks
Thomas


--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






On Wed, Oct 5, 2011 at 9:07 AM, Syed Haider <[hidden email]> wrote:

Thomas,

Converting your source database into a mart is not mandatory, its only recommended for performance reasons. Try using the source schema & create a dataset straight out of your source database (without materialising to mart) and the subsequent querying and web interfaces should work just fine (at least in theory).

W.r.t your second question, Martconfigurator is not able to read GFF or other text file formats. The input to Martconfigurator should either be an existing biomart webserver end point or a database.

HTH,
Syed



On 04/10/2011 22:03, Thomas TRIPLET wrote:
Hello,

I'm trying to build a mart (v0.8rc6) from a normalized (3NF) PostgreSQL database. So far, it looks really great, except for one issue I have. When I look at the source database schema (using MartConfigurator), it is just fine, so the database seems properly imported. However, the corresponding target database really messes things up as it tries to merge everything together. Here is a simple example to illustrate the problem:

[cid:ii_132d0b1d058ca382]  [cid:ii_132d0b234e4777ae]



The target database is clearly problematic, and not usable as such. Is there a way to force BioMart to keep the source schema untouched, or at least, control how it is processed ? ALl relations in the source database are defined as [1:M] so I'm not sure what's wrong here.

Also, it is my understanding that a database must exist in the form of a database in order to be imported (whether it is actually accessed using RDBMS/URL/registry), or can it also be defined as raw datafiles (e.g GFF files) along with a parser. Could you please confirm this?

Thanks a lot for your help.

Thomas

--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






_______________________________________________
Users mailing list
[hidden email]
https://lists.biomart.org/mailman/listinfo/users









_______________________________________________
Users mailing list
[hidden email]
https://lists.biomart.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: [BioMart Users] Problem building 3NF virtual mart

Joachim Baran-2
Hello Thomas,

On 11-10-05 4:41 PM, "Arek Kasprzyk" <[hidden email]> wrote:
The target schema will be a query optimized version of the source schema. The target schema complies with the reverse star spec so it will always be different from the source. However, the schemas and SQL against them are functionally equivalent so they will return the same data.
  As Arek explained it here, there is no need to worry about how the data is materialized. Queries on a materialized mart will return the same results as queries on a virtual mart, but the former are faster due to the nature of how query engines work. Data "denormalization" into reverse star- or snowflake-schema is quite common when handling read-only databases. The nice thing about BioMart is the fact that it does the data denormalization for you — so you do not really need to know what happens behind the curtains in order to benefit from the resulting query optimizations.

  Anyway, there is currently a whole bunch of BioMart papers that have been published in Database Journal, which might help you to get a bit more into the details how the system works: http://database.oxfordjournals.org/search?submit=yes&fulltext=biomart

Joachim

Joachim Baran, PhD
Database Programmer/Technical Analyst, Informatics and Bio-computing

Ontario Institute for Cancer Research
MaRS Centre, South Tower

101 College Street, Suite 800

Toronto, Ontario, Canada M5G 0A3


Tel: 647-258-4319

Toll-free: 1-866-678-6427

www.oicr.on.ca


This message and any attachments may contain confidential and/or privileged information for the sole use of the intended recipient. Any review or distribution by anyone other than the person for whom it was originally intended is strictly prohibited. If you have received this message in error, please contact the sender and delete all copies. Opinions, conclusions or other information contained in this message may not be that of the organization.

 



_______________________________________________
Users mailing list
[hidden email]
https://lists.biomart.org/mailman/listinfo/users
Reply | Threaded
Open this post in threaded view
|

Re: [BioMart Users] Problem building 3NF virtual mart

Thomas TRIPLET
In reply to this post by Arek Kasprzyk
Hi Arek,
I'm getting there, I managed to change the UI to fit my needs. I'm actually testing a few tools, which is why I've got specific questions, but so far, BioMart is really outstanding!

Just a last question, how can I activate the logs for the web UI to see the SQL queries that are generated in the backend. I've set DEBUG to ON indist/biomart.properties, but the dist/logs folder doesn't get populated (I've restarted the servers).

Thanks a lot!
Thomas

--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






On Wed, Oct 5, 2011 at 4:41 PM, Arek Kasprzyk <[hidden email]> wrote:
Hi Thomas,
I am not entirely sure what you mean by 'MartConfigurator can't control how the target schema is generated'. MConfigurator executes the 3NF schema transformation algorithm that i designed while still at EBI and that has been described in the recent paper:

http://database.oxfordjournals.org/content/2011/bar038.full.pdf+html

The target schema will be a query optimized version of the source schema. The target schema complies with the reverse star spec so it will always be different from the source. However, the schemas and SQL against them are functionally equivalent so they will return the same data.

Please let me know if you need more information
a



On Wed, Oct 5, 2011 at 4:05 PM, Thomas TRIPLET <[hidden email]> wrote:
Hi Arek,

Thanks for your message, that's definitely  very helpful. Still got a question about the target database as I am trying to think in the longer term. At some point, as more data comes in, I will probably have to materialize the virtual mart as recommended in the documentation, so that queries are executed in a reasonable amount of time. I'm not there yet, but at that point, it will be the target database that will be used (since it is the one that is materialized). So my initial question regarding the target schema still holds for longer term perspectives. If MartConfigurator can't control how the target schema is generated, is it possible to manually edit the underlying registry file so that the source and target schema match?

Also, this might sound a bit stupid, but how do I activate the logs for the web UI. I've set DEBUG to ON in dist/biomart.properties, but the dist/logs folder doesn't get populated (I've restarted the servers).

Thanks!
Thomas

--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






On Wed, Oct 5, 2011 at 1:45 PM, Arek Kasprzyk <[hidden email]> wrote:
Hi Thomas,
sorry, i did not make myself clear. The GUI gets configured as if it was talking to the target schema. However, the QueryComplier queries the source not target. The target simply does not physically exist at this point unless you materilize it. If you want to make sure that the source is being queried you can switch the logging on to see the actual SQL that is being compiled. As far as the GUI is concerned you can use MConfigurator to give it any shape of form that you require. It does not have to look like an image of the target schema. This is just a default behaviour to get you started

a


On Wed, Oct 5, 2011 at 1:25 PM, Thomas TRIPLET <[hidden email]> wrote:
Hi Arek,

I tried to add only the tables from my previous example and started the servers without materializing them.
It seems that the QueryCompiler does not ignore the target schema (cf screenshot: http://ttriplet.fg.concordia.ca/literatureTrial.png).

As far as I can tell with such a simple example, querying works in the sense that data is retrieved, but the forms and filters that must be populated for querying are not intuitive nor practical.

Also, it might be a related fact/error, when I import the 4 tables altogether, 2 data sources are created (one called literature, one person). I noticed this usually happens when the FKs in the source schema are not properly defined, but this is not the case here.

Also, the log gave me the following when I imported the tables:

org.biomart.common.exceptions.ValidationException: You can have at most one each of 1:M and M:1 subclass relations from the same table, but no more.
  at org.biomart.objects.objects.Relation.setSubclassRelation(Relation.java:447)
  at org.biomart.configurator.controller.MartController.continueSubclassing(MartController.java:1562)
  at org.biomart.configurator.controller.MartController.suggestMarts(MartController.java:1367)
  at org.biomart.configurator.controller.MartController.requestCreateMartsFromSource(MartController.java:1599)
  at org.biomart.configurator.controller.ObjectController.initMarts(ObjectController.java:98)
  at org.biomart.configurator.view.component.container.SourceGroupPanel$1.construct(SourceGroupPanel.java:85)
  at org.biomart.common.view.gui.SwingWorker$2.run(SwingWorker.java:128)
  at java.lang.Thread.run(Thread.java:679)

So it looks like the problem is that one table has two 1:M relations (probably the table used to define the M:N relation between articles and authors).
Thanks a lot for your help.

--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






On Wed, Oct 5, 2011 at 12:31 PM, Arek Kasprzyk <[hidden email]> wrote:
Hi Thomas,
What happens if you launch the server and query it? If you do not materilize, the QueryCompiler will ignore the target schema and translate SQL against the source instead.

Could you try querying it to see if you are getting the results back?

a



On Wed, Oct 5, 2011 at 11:51 AM, Thomas TRIPLET <[hidden email]> wrote:
Hi Arek,

This is because my source schema is normalized. When added as a RDBMS data source in BioMart, it is processed to build the target schema. The problem is that the target schema that is generated is denormalized and unusable. Here are 2 screenshots to illustrate the issue:
I hope this clarifies my problem.

Thanks
Thomas

--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






On Wed, Oct 5, 2011 at 11:39 AM, Arek Kasprzyk <[hidden email]> wrote:
Hi Thomas,
you lost me here. Why would you want to use the same schema for source and target? Could you elaborate on this please?

a

On Wed, Oct 5, 2011 at 11:24 AM, Thomas TRIPLET <[hidden email]> wrote:
Syed,
Thanks for your reply. Sorry I wasn't clear enough, I am trying to use the data source as it is, not materializing it as a mart. But when I add the data source as a Postgres RDBMS, MartConfigurator automatically uses the source schema to build the target schema. Is it possible to disable this feature and force MartConfigurator to use the same schema for the target database as for the source database?
Thanks
Thomas


--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






On Wed, Oct 5, 2011 at 9:07 AM, Syed Haider <[hidden email]> wrote:

Thomas,

Converting your source database into a mart is not mandatory, its only recommended for performance reasons. Try using the source schema & create a dataset straight out of your source database (without materialising to mart) and the subsequent querying and web interfaces should work just fine (at least in theory).

W.r.t your second question, Martconfigurator is not able to read GFF or other text file formats. The input to Martconfigurator should either be an existing biomart webserver end point or a database.

HTH,
Syed



On 04/10/2011 22:03, Thomas TRIPLET wrote:
Hello,

I'm trying to build a mart (v0.8rc6) from a normalized (3NF) PostgreSQL database. So far, it looks really great, except for one issue I have. When I look at the source database schema (using MartConfigurator), it is just fine, so the database seems properly imported. However, the corresponding target database really messes things up as it tries to merge everything together. Here is a simple example to illustrate the problem:

[cid:ii_132d0b1d058ca382]  [cid:ii_132d0b234e4777ae]



The target database is clearly problematic, and not usable as such. Is there a way to force BioMart to keep the source schema untouched, or at least, control how it is processed ? ALl relations in the source database are defined as [1:M] so I'm not sure what's wrong here.

Also, it is my understanding that a database must exist in the form of a database in order to be imported (whether it is actually accessed using RDBMS/URL/registry), or can it also be defined as raw datafiles (e.g GFF files) along with a parser. Could you please confirm this?

Thanks a lot for your help.

Thomas

--
Thomas Triplet, Ph.D.
http://www.thomastriplet.net

Centre for Structural and Functional Genomics
Concordia University
7141 West Sherbrooke St
Montreal QC H4B 1R6






_______________________________________________
Users mailing list
[hidden email]
https://lists.biomart.org/mailman/listinfo/users










_______________________________________________
Users mailing list
[hidden email]
https://lists.biomart.org/mailman/listinfo/users