Re: Building tool dependencies

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Building tool dependencies

Matthias De Smet

>
> Hi
>
> I’m trying to put together a tool + dependency package to use umi-tools (https://github.com/CGATOxford/UMI-tools) from Galaxy.
> I’m at a loss on how I should organise this.
>
> The package is available from either github or pip. Which one is preferable to use? How do I  make sure the package dependencies are installed? I’ve found the <action type="download_by_url”> and <action type=“create_virtualenv”> actions, but I’m not really sure that’s the way to go. I havent found any real resources on how to build package dependencies either.
>
> Any help? Now, approaching it by downloading the package from github with <action type="download_by_url”> and installing it with <action type="shell_command">python setup.py install</action>. This, however, tries to install the package to /usr/bin/ which isn’t desirable (obviously).
>
> Thanks a lot!
> M
>

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Building tool dependencies

Peter van Heusden
Actually, for recent Galaxy, I'd recommend making a conda recipe for UMI-tools and contributing it to bioconda:


Then your tool can just depend on this and the conda dependency resolver (default in Galaxy since 16.01) can find it. Others can comment on the recent work on conda/toolshed integration.

Peter

On 5 July 2016 at 08:56, Matthias De Smet <[hidden email]> wrote:

>
> Hi
>
> I’m trying to put together a tool + dependency package to use umi-tools (https://github.com/CGATOxford/UMI-tools) from Galaxy.
> I’m at a loss on how I should organise this.
>
> The package is available from either github or pip. Which one is preferable to use? How do I  make sure the package dependencies are installed? I’ve found the <action type="download_by_url”> and <action type=“create_virtualenv”> actions, but I’m not really sure that’s the way to go. I havent found any real resources on how to build package dependencies either.
>
> Any help? Now, approaching it by downloading the package from github with <action type="download_by_url”> and installing it with <action type="shell_command">python setup.py install</action>. This, however, tries to install the package to /usr/bin/ which isn’t desirable (obviously).
>
> Thanks a lot!
> M
>

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Building tool dependencies

Peter Briggs
In reply to this post by Matthias De Smet
Hello Matthias

 From what I've seen there are a number of different recipes used for
installing Python packages via the toolshed; the approach you're using
seems to be pretty common. In these cases the usual 'shell_command'
seems to be:

python setup.py install --install-lib $INSTALL_DIR/lib/python
--install-scripts $INSTALL_DIR/bin

which should avoid installing to /usr/bin.

You'll then need to do something like:

<action type="set_environment">
   <environment_variable action="prepend_to"
name="PYTHONPATH">$INSTALL_DIR/lib/python</environment_variable>
   <environment_variable action="prepend_to"
name="PATH">$INSTALL_DIR/bin</environment_variable>
   ...
</action>

to ensure that the correct environment is set up when the tool is used.

Aside from this: elsewhere I've seen other approaches including a
"magic" "package" command where the whole download and install is taken
care of in a single tag. For example:

...
<action type="setup_python_environment">
   <repository name="package_python_2_7_10" owner="iuc">
      <package name="python" version="2.7.10" />
    </repository>
    <package
sha256sum="e01853dfe111f3aea005315573400b7216ddbabbf1f28d482a71217d67ae4f81">https://pypi.python.org/packages/source/p/pandas/pandas-0.16.2.tar.gz</package>
...

(see
https://github.com/galaxyproject/tools-iuc/blob/master/packages/package_python_2_7_pandas_0_16/tool_dependencies.xml 
for the full recipe that I've pulled this fragment from.)

HTH

Best wishes

Peter

On 05/07/16 07:56, Matthias De Smet wrote:

>
>>
>> Hi
>>
>> I’m trying to put together a tool + dependency package to use umi-tools (https://github.com/CGATOxford/UMI-tools) from Galaxy.
>> I’m at a loss on how I should organise this.
>>
>> The package is available from either github or pip. Which one is preferable to use? How do I  make sure the package dependencies are installed? I’ve found the <action type="download_by_url”> and <action type=“create_virtualenv”> actions, but I’m not really sure that’s the way to go. I havent found any real resources on how to build package dependencies either.
>>
>> Any help? Now, approaching it by downloading the package from github with <action type="download_by_url”> and installing it with <action type="shell_command">python setup.py install</action>. This, however, tries to install the package to /usr/bin/ which isn’t desirable (obviously).
>>
>> Thanks a lot!
>> M
>>
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>    https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>    http://galaxyproject.org/search/mailinglists/
>

--
Peter Briggs [hidden email]
Bioinformatics Core Facility University of Manchester
B.1083 Michael Smith Bldg Tel: (0161) 2751482
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: Building tool dependencies

Matthias De Smet
In reply to this post by Matthias De Smet
Hi All!

Thanks for chiming in! I’ll give the conda recipe approach a go, this seems to be the easiest way to go, and the most futureproof!

Thanks!

M

> Message: 1
> Date: Tue, 5 Jul 2016 06:56:00 +0000
> From: Matthias De Smet <[hidden email]>
> To: "[hidden email]"
> <[hidden email]>
> Subject: Re: [galaxy-dev] Building tool dependencies
> Message-ID: <[hidden email]>
> Content-Type: text/plain; charset="utf-8"
>
>
>>
>> Hi
>>
>> I’m trying to put together a tool + dependency package to use umi-tools (https://github.com/CGATOxford/UMI-tools) from Galaxy.
>> I’m at a loss on how I should organise this.
>>
>> The package is available from either github or pip. Which one is preferable to use? How do I  make sure the package dependencies are installed? I’ve found the <action type="download_by_url”> and <action type=“create_virtualenv”> actions, but I’m not really sure that’s the way to go. I havent found any real resources on how to build package dependencies either.
>>
>> Any help? Now, approaching it by downloading the package from github with <action type="download_by_url”> and installing it with <action type="shell_command">python setup.py install</action>. This, however, tries to install the package to /usr/bin/ which isn’t desirable (obviously).
>>
>> Thanks a lot!
>> M
>>
>
>
> ------------------------------
>
> Message: 2
> Date: Tue, 5 Jul 2016 09:33:09 +0200
> From: Peter van Heusden <[hidden email]>
> To: [hidden email]
> Subject: Re: [galaxy-dev] Building tool dependencies
> Message-ID:
> <[hidden email]>
> Content-Type: text/plain; charset="utf-8"
>
> Actually, for recent Galaxy, I'd recommend making a conda recipe for
> UMI-tools and contributing it to bioconda:
>
> https://bioconda.github.io/
>
> Then your tool can just depend on this and the conda dependency resolver
> (default in Galaxy since 16.01) can find it. Others can comment on the
> recent work on conda/toolshed integration.
>
> Peter
>
> On 5 July 2016 at 08:56, Matthias De Smet <[hidden email]> wrote:
>
>>
>>>
>>> Hi
>>>
>>> I’m trying to put together a tool + dependency package to use umi-tools (
>> https://github.com/CGATOxford/UMI-tools) from Galaxy.
>>> I’m at a loss on how I should organise this.
>>>
>>> The package is available from either github or pip. Which one is
>> preferable to use? How do I  make sure the package dependencies are
>> installed? I’ve found the <action type="download_by_url”> and <action
>> type=“create_virtualenv”> actions, but I’m not really sure that’s the way
>> to go. I havent found any real resources on how to build package
>> dependencies either.
>>>
>>> Any help? Now, approaching it by downloading the package from github
>> with <action type="download_by_url”> and installing it with <action
>> type="shell_command">python setup.py install</action>. This, however, tries
>> to install the package to /usr/bin/ which isn’t desirable (obviously).
>>>
>>> Thanks a lot!
>>> M
>>>
>>
>> ___________________________________________________________
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>  https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>  http://galaxyproject.org/search/mailinglists/
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <https://lists.galaxyproject.org/pipermail/galaxy-dev/attachments/20160705/0b422d9d/attachment-0001.html>
>
> ------------------------------
>
> Message: 3
> Date: Tue, 5 Jul 2016 08:49:54 +0100
> From: Peter Briggs <[hidden email]>
> To: [hidden email]
> Subject: Re: [galaxy-dev] Building tool dependencies
> Message-ID: <[hidden email]>
> Content-Type: text/plain; charset=utf-8; format=flowed
>
> Hello Matthias
>
> From what I've seen there are a number of different recipes used for
> installing Python packages via the toolshed; the approach you're using
> seems to be pretty common. In these cases the usual 'shell_command'
> seems to be:
>
> python setup.py install --install-lib $INSTALL_DIR/lib/python
> --install-scripts $INSTALL_DIR/bin
>
> which should avoid installing to /usr/bin.
>
> You'll then need to do something like:
>
> <action type="set_environment">
>   <environment_variable action="prepend_to"
> name="PYTHONPATH">$INSTALL_DIR/lib/python</environment_variable>
>   <environment_variable action="prepend_to"
> name="PATH">$INSTALL_DIR/bin</environment_variable>
>   ...
> </action>
>
> to ensure that the correct environment is set up when the tool is used.
>
> Aside from this: elsewhere I've seen other approaches including a
> "magic" "package" command where the whole download and install is taken
> care of in a single tag. For example:
>
> ...
> <action type="setup_python_environment">
>   <repository name="package_python_2_7_10" owner="iuc">
>      <package name="python" version="2.7.10" />
>    </repository>
>    <package
> sha256sum="e01853dfe111f3aea005315573400b7216ddbabbf1f28d482a71217d67ae4f81">https://pypi.python.org/packages/source/p/pandas/pandas-0.16.2.tar.gz</package>
> ...
>
> (see
> https://github.com/galaxyproject/tools-iuc/blob/master/packages/package_python_2_7_pandas_0_16/tool_dependencies.xml 
> for the full recipe that I've pulled this fragment from.)
>
> HTH
>
> Best wishes
>
> Peter
>
> On 05/07/16 07:56, Matthias De Smet wrote:
>>
>>>
>>> Hi
>>>
>>> I’m trying to put together a tool + dependency package to use umi-tools (https://github.com/CGATOxford/UMI-tools) from Galaxy.
>>> I’m at a loss on how I should organise this.
>>>
>>> The package is available from either github or pip. Which one is preferable to use? How do I  make sure the package dependencies are installed? I’ve found the <action type="download_by_url”> and <action type=“create_virtualenv”> actions, but I’m not really sure that’s the way to go. I havent found any real resources on how to build package dependencies either.
>>>
>>> Any help? Now, approaching it by downloading the package from github with <action type="download_by_url”> and installing it with <action type="shell_command">python setup.py install</action>. This, however, tries to install the package to /usr/bin/ which isn’t desirable (obviously).
>>>
>>> Thanks a lot!
>>> M
>>>
>>
>> ___________________________________________________________
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>   https://lists.galaxyproject.org/
>>
>> To search Galaxy mailing lists use the unified search at:
>>   http://galaxyproject.org/search/mailinglists/
>>
>
> --
> Peter Briggs [hidden email]
> Bioinformatics Core Facility University of Manchester
> B.1083 Michael Smith Bldg Tel: (0161) 2751482

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/