"Merging tabular data tables with non-matching columns is not allowed"

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

"Merging tabular data tables with non-matching columns is not allowed"

Peter van Heusden
Hi there

I'm working on a data manager to keep track of the reference data collection used by the SHAPEIT2 tool. I had to add a column to the data manager but when I try and do a `planemo shed_update` I get the error:

Unexpected HTTP status code: 500: {"err_msg": "Error attempting to parse file tool_data_table_conf.xml.sample: Merging tabular data tables with non matching columns is not allowed: shapeit_ref:{'path': 2, 'reference_prefix': 3, 'name': 1, 'value': 0, 'map_prefix': 4} != shapeit_ref:{'name': 1, 'value': 0, 'sample_prefix': 5, 'path': 2, 'reference_prefix': 3, 'map_prefix': 4}"}

This is for data_manager_shapeit_reference in testtoolshed, which is on github at


As far as I can tell the columns do accord. Is there any way to see which files are being merged to help try and debug this error?

Thanks!
Peter

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
Reply | Threaded
Open this post in threaded view
|

Re: "Merging tabular data tables with non-matching columns is not allowed"

Peter van Heusden
I'm following up on my previous email. I somehow managed to get `planemo shed_update` to work but now I seem to have an inconsistent view of the Test toolshed. This is how it looks from the Admin panel of my Galaxy server:

https://files.gitter.im/galaxyproject/Lobby/k2qH/image.png
Notice that it shows 3 installable revisions for the data manager, last updated 13th of September. And this is how it looks from the Test toolshed website:

https://files.gitter.im/galaxyproject/Lobby/ymY9/image.png


Notice that it has 6 installabale revisions for the data manager. Is my Galaxy server perhaps caching something?

Peter


On Sat, 15 Sep 2018 at 09:57 Peter van Heusden <[hidden email]> wrote:
Hi there

I'm working on a data manager to keep track of the reference data collection used by the SHAPEIT2 tool. I had to add a column to the data manager but when I try and do a `planemo shed_update` I get the error:

Unexpected HTTP status code: 500: {"err_msg": "Error attempting to parse file tool_data_table_conf.xml.sample: Merging tabular data tables with non matching columns is not allowed: shapeit_ref:{'path': 2, 'reference_prefix': 3, 'name': 1, 'value': 0, 'map_prefix': 4} != shapeit_ref:{'name': 1, 'value': 0, 'sample_prefix': 5, 'path': 2, 'reference_prefix': 3, 'map_prefix': 4}"}

This is for data_manager_shapeit_reference in testtoolshed, which is on github at


As far as I can tell the columns do accord. Is there any way to see which files are being merged to help try and debug this error?

Thanks!
Peter

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
Reply | Threaded
Open this post in threaded view
|

Re: "Merging tabular data tables with non-matching columns is not allowed"

Nicola Soranzo-3
Hi Peter,
if you open the page for data_manager_shapeit_reference in the test ToolShed you will see that revision 5 and 6 are not installable, they both have "This revision can be installed: False" . If you instead select revision 3 it is installable.

Cheers,
Nicola

On 17/09/18 11:51, Peter van Heusden wrote:
I'm following up on my previous email. I somehow managed to get `planemo shed_update` to work but now I seem to have an inconsistent view of the Test toolshed. This is how it looks from the Admin panel of my Galaxy server:

https://files.gitter.im/galaxyproject/Lobby/k2qH/image.png
Notice that it shows 3 installable revisions for the data manager, last updated 13th of September. And this is how it looks from the Test toolshed website:

https://files.gitter.im/galaxyproject/Lobby/ymY9/image.png


Notice that it has 6 installabale revisions for the data manager. Is my Galaxy server perhaps caching something?

Peter


On Sat, 15 Sep 2018 at 09:57 Peter van Heusden <[hidden email]> wrote:
Hi there

I'm working on a data manager to keep track of the reference data collection used by the SHAPEIT2 tool. I had to add a column to the data manager but when I try and do a `planemo shed_update` I get the error:

Unexpected HTTP status code: 500: {"err_msg": "Error attempting to parse file tool_data_table_conf.xml.sample: Merging tabular data tables with non matching columns is not allowed: shapeit_ref:{'path': 2, 'reference_prefix': 3, 'name': 1, 'value': 0, 'map_prefix': 4} != shapeit_ref:{'name': 1, 'value': 0, 'sample_prefix': 5, 'path': 2, 'reference_prefix': 3, 'map_prefix': 4}"}

This is for data_manager_shapeit_reference in testtoolshed, which is on github at


As far as I can tell the columns do accord. Is there any way to see which files are being merged to help try and debug this error?

Thanks!
Peter


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
Reply | Threaded
Open this post in threaded view
|

Re: "Merging tabular data tables with non-matching columns is not allowed"

Peter van Heusden
Thanks - which is probably because of the previous failures with `shed_update` - which relate back to the topic of my original email. I suspect that failure from `planemo shed_update` has led to this "not installable" state. And I don't know how to solve the problem from `shed_update` which *looks* like it is coming from a clash between previous and current definitions of the tool-data table.

Peter

On Mon, 17 Sep 2018 at 12:58 Nicola Soranzo <[hidden email]> wrote:
Hi Peter,
if you open the page for data_manager_shapeit_reference in the test ToolShed you will see that revision 5 and 6 are not installable, they both have "This revision can be installed: False" . If you instead select revision 3 it is installable.

Cheers,
Nicola


On 17/09/18 11:51, Peter van Heusden wrote:
I'm following up on my previous email. I somehow managed to get `planemo shed_update` to work but now I seem to have an inconsistent view of the Test toolshed. This is how it looks from the Admin panel of my Galaxy server:

https://files.gitter.im/galaxyproject/Lobby/k2qH/image.png
Notice that it shows 3 installable revisions for the data manager, last updated 13th of September. And this is how it looks from the Test toolshed website:

https://files.gitter.im/galaxyproject/Lobby/ymY9/image.png


Notice that it has 6 installabale revisions for the data manager. Is my Galaxy server perhaps caching something?

Peter


On Sat, 15 Sep 2018 at 09:57 Peter van Heusden <[hidden email]> wrote:
Hi there

I'm working on a data manager to keep track of the reference data collection used by the SHAPEIT2 tool. I had to add a column to the data manager but when I try and do a `planemo shed_update` I get the error:

Unexpected HTTP status code: 500: {"err_msg": "Error attempting to parse file tool_data_table_conf.xml.sample: Merging tabular data tables with non matching columns is not allowed: shapeit_ref:{'path': 2, 'reference_prefix': 3, 'name': 1, 'value': 0, 'map_prefix': 4} != shapeit_ref:{'name': 1, 'value': 0, 'sample_prefix': 5, 'path': 2, 'reference_prefix': 3, 'map_prefix': 4}"}

This is for data_manager_shapeit_reference in testtoolshed, which is on github at


As far as I can tell the columns do accord. Is there any way to see which files are being merged to help try and debug this error?

Thanks!
Peter


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
Reply | Threaded
Open this post in threaded view
|

Re: "Merging tabular data tables with non-matching columns is not allowed"

Peter van Heusden
A bit more tracing leads me to suspect that what is happening is that a merge is being called here:


from:


i.e. the toolshed maintains an in-memory copy of tool-data tables and if the list of columns in an incoming tool-data table is not the same as the in-memory version the error I saw happens.

Which leaves me with the question: how is one meant to update the column list of a tool data table?

Peter

On Mon, 17 Sep 2018 at 13:23 Peter van Heusden <[hidden email]> wrote:
Thanks - which is probably because of the previous failures with `shed_update` - which relate back to the topic of my original email. I suspect that failure from `planemo shed_update` has led to this "not installable" state. And I don't know how to solve the problem from `shed_update` which *looks* like it is coming from a clash between previous and current definitions of the tool-data table.

Peter

On Mon, 17 Sep 2018 at 12:58 Nicola Soranzo <[hidden email]> wrote:
Hi Peter,
if you open the page for data_manager_shapeit_reference in the test ToolShed you will see that revision 5 and 6 are not installable, they both have "This revision can be installed: False" . If you instead select revision 3 it is installable.

Cheers,
Nicola


On 17/09/18 11:51, Peter van Heusden wrote:
I'm following up on my previous email. I somehow managed to get `planemo shed_update` to work but now I seem to have an inconsistent view of the Test toolshed. This is how it looks from the Admin panel of my Galaxy server:

https://files.gitter.im/galaxyproject/Lobby/k2qH/image.png
Notice that it shows 3 installable revisions for the data manager, last updated 13th of September. And this is how it looks from the Test toolshed website:

https://files.gitter.im/galaxyproject/Lobby/ymY9/image.png


Notice that it has 6 installabale revisions for the data manager. Is my Galaxy server perhaps caching something?

Peter


On Sat, 15 Sep 2018 at 09:57 Peter van Heusden <[hidden email]> wrote:
Hi there

I'm working on a data manager to keep track of the reference data collection used by the SHAPEIT2 tool. I had to add a column to the data manager but when I try and do a `planemo shed_update` I get the error:

Unexpected HTTP status code: 500: {"err_msg": "Error attempting to parse file tool_data_table_conf.xml.sample: Merging tabular data tables with non matching columns is not allowed: shapeit_ref:{'path': 2, 'reference_prefix': 3, 'name': 1, 'value': 0, 'map_prefix': 4} != shapeit_ref:{'name': 1, 'value': 0, 'sample_prefix': 5, 'path': 2, 'reference_prefix': 3, 'map_prefix': 4}"}

This is for data_manager_shapeit_reference in testtoolshed, which is on github at


As far as I can tell the columns do accord. Is there any way to see which files are being merged to help try and debug this error?

Thanks!
Peter


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
Reply | Threaded
Open this post in threaded view
|

Re: "Merging tabular data tables with non-matching columns is not allowed"

Peter Cock
Hi Peter vH,

I wonder if adding columns is not supported - because some tools
would define the table without the extra column? You might have
to define a new data table instead (with largely the same content).
But to be clear, I am not an expert in this area of Galaxy.

However, it does seem the IUC could lay out some *.loc file best
practises for tool authors. There is currently a TODO entry here:


Peter C.

On Mon, Sep 17, 2018 at 10:56 PM Peter van Heusden <[hidden email]> wrote:
A bit more tracing leads me to suspect that what is happening is that a merge is being called here:


from:


i.e. the toolshed maintains an in-memory copy of tool-data tables and if the list of columns in an incoming tool-data table is not the same as the in-memory version the error I saw happens.

Which leaves me with the question: how is one meant to update the column list of a tool data table?

Peter



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
Reply | Threaded
Open this post in threaded view
|

Re: "Merging tabular data tables with non-matching columns is not allowed"

Peter van Heusden
Yes that was my conclusion, thanks Peter.

As a result I switched to a new table named (shapeit2_ref) and a new tool name (data_manager_shapeit2_reference) and from planemo side upload seemed to succeed but the tool is in an error state. I resorted to manually editing the tool-data table.

Data managers are a bit of an ugly duckling part of Galaxy - not receiving as much love as other kinds of tools.

Peter


On Tue, 18 Sep 2018 at 14:22 Peter Cock <[hidden email]> wrote:
Hi Peter vH,

I wonder if adding columns is not supported - because some tools
would define the table without the extra column? You might have
to define a new data table instead (with largely the same content).
But to be clear, I am not an expert in this area of Galaxy.

However, it does seem the IUC could lay out some *.loc file best
practises for tool authors. There is currently a TODO entry here:


Peter C.

On Mon, Sep 17, 2018 at 10:56 PM Peter van Heusden <[hidden email]> wrote:
A bit more tracing leads me to suspect that what is happening is that a merge is being called here:


from:


i.e. the toolshed maintains an in-memory copy of tool-data tables and if the list of columns in an incoming tool-data table is not the same as the in-memory version the error I saw happens.

Which leaves me with the question: how is one meant to update the column list of a tool data table?

Peter



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/