Changing Tool Shed URL

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Changing Tool Shed URL

Youssef  GHORBAL
Hello,

        I’m wondering if there is any way to change an established Tool Shed URL.
        I have a Galaxy instance and a Tool Shed instance (http://mytoolshed.url/xxx) A lot of tools have beed installed from that Tool Shed inside Galaxy (tools referenced inside workflows etc)
        I want to change the Tool Shed URL from http://mytoolshed.url/xxx to http://mytoolshed2.url/yyyy.

        Galaxy and Tool Shed seems to use parts of the URL as a guid for tools.
       
        Tool Shed does not seem to bother to be called by either URLs (it generates guids with the new URLs scheam for new tools though)

        Galaxy does not seem to handle it, I changed the tool_sheds_conf.xml to point to new URL, but when I update a tool on the Tool Shed, Galaxy gets the new version but it generates buggy tool panel (the tool is cited twice)
        I also tried to mess around with the shed_tool_conf.xml apdapting the guids and the <tool_shed> items with no luck.

        Can someone point me to the right direction ? How the matching is done between shed_tool_conf.xml entries, the tool_version and tool_shed_repository tables.

        Thank you for your help.

Youssef Ghorbal
Institut Pasteur
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Changing Tool Shed URL

Marius van den Beek
Hi Youssef,

the tool shed url is also stored in the database, that's why just changing
the shed_tool_conf.xml is not working. I think you have a bunch of options here:
1: You can make a list of tools from that toolshed and re-install them from the new toolshed.
https://pypi.python.org/pypi/ephemeris/ is a nice tool to do this efficiently.
Galaxy is (or should be) smart enough to recognize this and will (should) not break existing workflows and tool reruns.
2: You can setup an alias in your /etc/hosts file if only the toolshed's URL has changed, or you can setup a proxy to forward the requests
if also the subdirectory has changed.
3. You can try to manipulate the database directly. (But I would seriously advise you to go with possibilities 1 or 2).

Best,
Marius

On 20 October 2016 at 00:16, Youssef GHORBAL <[hidden email]> wrote:
Hello,

        I’m wondering if there is any way to change an established Tool Shed URL.
        I have a Galaxy instance and a Tool Shed instance (http://mytoolshed.url/xxx) A lot of tools have beed installed from that Tool Shed inside Galaxy (tools referenced inside workflows etc)
        I want to change the Tool Shed URL from http://mytoolshed.url/xxx to http://mytoolshed2.url/yyyy.

        Galaxy and Tool Shed seems to use parts of the URL as a guid for tools.

        Tool Shed does not seem to bother to be called by either URLs (it generates guids with the new URLs scheam for new tools though)

        Galaxy does not seem to handle it, I changed the tool_sheds_conf.xml to point to new URL, but when I update a tool on the Tool Shed, Galaxy gets the new version but it generates buggy tool panel (the tool is cited twice)
        I also tried to mess around with the shed_tool_conf.xml apdapting the guids and the <tool_shed> items with no luck.

        Can someone point me to the right direction ? How the matching is done between shed_tool_conf.xml entries, the tool_version and tool_shed_repository tables.

        Thank you for your help.

Youssef Ghorbal
Institut Pasteur
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Changing Tool Shed URL

Olivia Doppelt-Azeroual

Hello Marius,

Thank you for your answer.

Are you sure that ephemeris can transfer tool from toolshed1 to toolshed2 ? We read that it was only implemented to install tools from a toolshed on a Galaxy.

We will try the second solution and as we already tried to manipulate the database, we stongly agree that it is not THE solution.

Cheers,

-

Fabien and Olivia and Youssef



Le 20/10/2016 à 10:29, Marius van den Beek a écrit :
Hi Youssef,

the tool shed url is also stored in the database, that's why just changing
the shed_tool_conf.xml is not working. I think you have a bunch of options here:
1: You can make a list of tools from that toolshed and re-install them from the new toolshed.
https://pypi.python.org/pypi/ephemeris/ is a nice tool to do this efficiently.
Galaxy is (or should be) smart enough to recognize this and will (should) not break existing workflows and tool reruns.
2: You can setup an alias in your /etc/hosts file if only the toolshed's URL has changed, or you can setup a proxy to forward the requests
if also the subdirectory has changed.
3. You can try to manipulate the database directly. (But I would seriously advise you to go with possibilities 1 or 2).

Best,
Marius

On 20 October 2016 at 00:16, Youssef GHORBAL <[hidden email]> wrote:
Hello,

        I’m wondering if there is any way to change an established Tool Shed URL.
        I have a Galaxy instance and a Tool Shed instance (http://mytoolshed.url/xxx) A lot of tools have beed installed from that Tool Shed inside Galaxy (tools referenced inside workflows etc)
        I want to change the Tool Shed URL from http://mytoolshed.url/xxx to http://mytoolshed2.url/yyyy.

        Galaxy and Tool Shed seems to use parts of the URL as a guid for tools.

        Tool Shed does not seem to bother to be called by either URLs (it generates guids with the new URLs scheam for new tools though)

        Galaxy does not seem to handle it, I changed the tool_sheds_conf.xml to point to new URL, but when I update a tool on the Tool Shed, Galaxy gets the new version but it generates buggy tool panel (the tool is cited twice)
        I also tried to mess around with the shed_tool_conf.xml apdapting the guids and the <tool_shed> items with no luck.

        Can someone point me to the right direction ? How the matching is done between shed_tool_conf.xml entries, the tool_version and tool_shed_repository tables.

        Thank you for your help.

Youssef Ghorbal
Institut Pasteur
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

-- 
Olivia Doppelt-Azeroual, PhD
Bioinformatics Engineer
CIB - C3BI - Institut Pasteur, Paris

Email: [hidden email]
Tel: 01 44 38 92 15

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Changing Tool Shed URL

Marius van den Beek

This wouldn’t be a transfer, you would uninstall the tools coming from the old toolshed and re-install them form the new toolshed.
This is only possible if these tools are available from the second toolshed.

To illustrate the procedure, this is what you would do if you would take all tools from usegalaxy.org,
and replace the tools coming from the main tool shed with the same tools but coming form the testtoolshed

get-tool-list -g https://usegalaxy.org/ -o tool_list.yml
sed -i .bak 's/toolshed.g2.bx.psu.edu/testtoolshed.g2.bx.psu.edu/g' tool_list.yml
shed-install -g <your_galaxy_instace> -a <your_api_key> -t tool_list.yml

On 20 October 2016 at 11:02, Olivia Doppelt-Azeroual <[hidden email]> wrote:

Hello Marius,

Thank you for your answer.

Are you sure that ephemeris can transfer tool from toolshed1 to toolshed2 ? We read that it was only implemented to install tools from a toolshed on a Galaxy.

We will try the second solution and as we already tried to manipulate the database, we stongly agree that it is not THE solution.

Cheers,

-

Fabien and Olivia and Youssef



Le 20/10/2016 à 10:29, Marius van den Beek a écrit :
Hi Youssef,

the tool shed url is also stored in the database, that's why just changing
the shed_tool_conf.xml is not working. I think you have a bunch of options here:
1: You can make a list of tools from that toolshed and re-install them from the new toolshed.
https://pypi.python.org/pypi/ephemeris/ is a nice tool to do this efficiently.
Galaxy is (or should be) smart enough to recognize this and will (should) not break existing workflows and tool reruns.
2: You can setup an alias in your /etc/hosts file if only the toolshed's URL has changed, or you can setup a proxy to forward the requests
if also the subdirectory has changed.
3. You can try to manipulate the database directly. (But I would seriously advise you to go with possibilities 1 or 2).

Best,
Marius

On 20 October 2016 at 00:16, Youssef GHORBAL <[hidden email]> wrote:
Hello,

        I’m wondering if there is any way to change an established Tool Shed URL.
        I have a Galaxy instance and a Tool Shed instance (http://mytoolshed.url/xxx) A lot of tools have beed installed from that Tool Shed inside Galaxy (tools referenced inside workflows etc)
        I want to change the Tool Shed URL from http://mytoolshed.url/xxx to http://mytoolshed2.url/yyyy.

        Galaxy and Tool Shed seems to use parts of the URL as a guid for tools.

        Tool Shed does not seem to bother to be called by either URLs (it generates guids with the new URLs scheam for new tools though)

        Galaxy does not seem to handle it, I changed the tool_sheds_conf.xml to point to new URL, but when I update a tool on the Tool Shed, Galaxy gets the new version but it generates buggy tool panel (the tool is cited twice)
        I also tried to mess around with the shed_tool_conf.xml apdapting the guids and the <tool_shed> items with no luck.

        Can someone point me to the right direction ? How the matching is done between shed_tool_conf.xml entries, the tool_version and tool_shed_repository tables.

        Thank you for your help.

Youssef Ghorbal
Institut Pasteur
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

-- 
Olivia Doppelt-Azeroual, PhD
Bioinformatics Engineer
CIB - C3BI - Institut Pasteur, Paris

Email: [hidden email]
Tel: 01 44 38 92 15


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Changing Tool Shed URL

Youssef  GHORBAL
In reply to this post by Youssef GHORBAL
Hello,
       
        What worked for us was this action sequence :
        - stop Galaxy
        - update config/tool_sheds_file.xml (change the URL from the old one to the new one)
        - “sed” shed_tool_conf.xml to change the “guid", the tool_shed and the “file” to replace the old prefix with the new prefix.
        - update the tool_version table (the tool_id column) to change the old prefix to the new prefix.
        - update the tool_shed_repositoriy table to change the tool_shed column to use the new tooshed prefix
        - update the directory shed_tools to rename the directory from the old prefix to the new prefix (care to adapt the subdirectories too : ./mytoolshed.url/xxx => ./mytoolshed2.url/yyyy)
        - start Galaxy (and cross your fingers)

        Hope this help !

Youssef Ghorbal
Institut Pasteur
-----------------

> On 20 Oct 2016, at 00:16, Youssef GHORBAL <[hidden email]> wrote:
>
> Hello,
>
> I’m wondering if there is any way to change an established Tool Shed URL.
> I have a Galaxy instance and a Tool Shed instance (http://mytoolshed.url/xxx) A lot of tools have beed installed from that Tool Shed inside Galaxy (tools referenced inside workflows etc)
> I want to change the Tool Shed URL from http://mytoolshed.url/xxx to http://mytoolshed2.url/yyyy.
>
> Galaxy and Tool Shed seems to use parts of the URL as a guid for tools.
>
> Tool Shed does not seem to bother to be called by either URLs (it generates guids with the new URLs scheam for new tools though)
>
> Galaxy does not seem to handle it, I changed the tool_sheds_conf.xml to point to new URL, but when I update a tool on the Tool Shed, Galaxy gets the new version but it generates buggy tool panel (the tool is cited twice)
> I also tried to mess around with the shed_tool_conf.xml apdapting the guids and the <tool_shed> items with no luck.
>
> Can someone point me to the right direction ? How the matching is done between shed_tool_conf.xml entries, the tool_version and tool_shed_repository tables.
>
> Thank you for your help.
>
> Youssef Ghorbal
> Institut Pasteur
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>  https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>  http://galaxyproject.org/search/mailinglists/

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Changing Tool Shed URL

Youssef  GHORBAL
Hello,

        Two other tables need to be adapated (they reference tool_id)
        Table “job" and table “workflow_step”

Youssef Ghorbal
Institut Pasteur
-----------------

> On 21 Oct 2016, at 00:36, Youssef Ghorbal <[hidden email]> wrote:
>
> Hello,
>
> What worked for us was this action sequence :
> - stop Galaxy
> - update config/tool_sheds_file.xml (change the URL from the old one to the new one)
> - “sed” shed_tool_conf.xml to change the “guid", the tool_shed and the “file” to replace the old prefix with the new prefix.
> - update the tool_version table (the tool_id column) to change the old prefix to the new prefix.
> - update the tool_shed_repositoriy table to change the tool_shed column to use the new tooshed prefix
> - update the directory shed_tools to rename the directory from the old prefix to the new prefix (care to adapt the subdirectories too : ./mytoolshed.url/xxx => ./mytoolshed2.url/yyyy)
> - start Galaxy (and cross your fingers)
>
> Hope this help !
>
> Youssef Ghorbal
> Institut Pasteur
> -----------------
>> On 20 Oct 2016, at 00:16, Youssef GHORBAL <[hidden email]> wrote:
>>
>> Hello,
>>
>> I’m wondering if there is any way to change an established Tool Shed URL.
>> I have a Galaxy instance and a Tool Shed instance (http://mytoolshed.url/xxx) A lot of tools have beed installed from that Tool Shed inside Galaxy (tools referenced inside workflows etc)
>> I want to change the Tool Shed URL from http://mytoolshed.url/xxx to http://mytoolshed2.url/yyyy.
>>
>> Galaxy and Tool Shed seems to use parts of the URL as a guid for tools.
>>
>> Tool Shed does not seem to bother to be called by either URLs (it generates guids with the new URLs scheam for new tools though)
>>
>> Galaxy does not seem to handle it, I changed the tool_sheds_conf.xml to point to new URL, but when I update a tool on the Tool Shed, Galaxy gets the new version but it generates buggy tool panel (the tool is cited twice)
>> I also tried to mess around with the shed_tool_conf.xml apdapting the guids and the <tool_shed> items with no luck.
>>
>> Can someone point me to the right direction ? How the matching is done between shed_tool_conf.xml entries, the tool_version and tool_shed_repository tables.
>>
>> Thank you for your help.
>>
>> Youssef Ghorbal
>> Institut Pasteur
>> ___________________________________________________________
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>> https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>> http://galaxyproject.org/search/mailinglists/
>

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Changing Tool Shed URL

Martin Čech
Thank you very much for sharing Youssef. Very helpful.

Martin

On Fri, Oct 21, 2016 at 10:20 AM Youssef GHORBAL <[hidden email]> wrote:
Hello,

        Two other tables need to be adapated (they reference tool_id)
        Table “job" and table “workflow_step”

Youssef Ghorbal
Institut Pasteur
-----------------
> On 21 Oct 2016, at 00:36, Youssef Ghorbal <[hidden email]> wrote:
>
> Hello,
>
>       What worked for us was this action sequence :
>       - stop Galaxy
>       - update config/tool_sheds_file.xml (change the URL from the old one to the new one)
>       - “sed” shed_tool_conf.xml to change the “guid", the tool_shed and the “file” to replace the old prefix with the new prefix.
>       - update the tool_version table (the tool_id column) to change the old prefix to the new prefix.
>       - update the tool_shed_repositoriy table to change the tool_shed column to use the new tooshed prefix
>       - update the directory shed_tools to rename the directory from the old prefix to the new prefix (care to adapt the subdirectories too : ./mytoolshed.url/xxx => ./mytoolshed2.url/yyyy)
>       - start Galaxy (and cross your fingers)
>
>       Hope this help !
>
> Youssef Ghorbal
> Institut Pasteur
> -----------------
>> On 20 Oct 2016, at 00:16, Youssef GHORBAL <[hidden email]> wrote:
>>
>> Hello,
>>
>>      I’m wondering if there is any way to change an established Tool Shed URL.
>>      I have a Galaxy instance and a Tool Shed instance (http://mytoolshed.url/xxx) A lot of tools have beed installed from that Tool Shed inside Galaxy (tools referenced inside workflows etc)
>>      I want to change the Tool Shed URL from http://mytoolshed.url/xxx to http://mytoolshed2.url/yyyy.
>>
>>      Galaxy and Tool Shed seems to use parts of the URL as a guid for tools.
>>
>>      Tool Shed does not seem to bother to be called by either URLs (it generates guids with the new URLs scheam for new tools though)
>>
>>      Galaxy does not seem to handle it, I changed the tool_sheds_conf.xml to point to new URL, but when I update a tool on the Tool Shed, Galaxy gets the new version but it generates buggy tool panel (the tool is cited twice)
>>      I also tried to mess around with the shed_tool_conf.xml apdapting the guids and the <tool_shed> items with no luck.
>>
>>      Can someone point me to the right direction ? How the matching is done between shed_tool_conf.xml entries, the tool_version and tool_shed_repository tables.
>>
>>      Thank you for your help.
>>
>> Youssef Ghorbal
>> Institut Pasteur
>> ___________________________________________________________
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>> https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>> http://galaxyproject.org/search/mailinglists/
>

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/