Tool to download galaxy datasets locally

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Tool to download galaxy datasets locally

evan clark
Has anyone attempted to write a tool that allows copying of galaxy
datasets locally, i.e. without downloading via http. We have scenario
where there are some additional tools available only via command line
and it would be very useful to be able to directly copy galaxy datasets
from the datasets folder.
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
Reply | Threaded
Open this post in threaded view
|

Re: Tool to download galaxy datasets locally

Nicola Soranzo-2
Hi Evan,
you can have a look at https://toolshed.g2.bx.psu.edu/view/earlhaminst/export_to_cluster/

Hope that helps!

Cheers,
Nicola

On 27/03/18 16:03, evan clark wrote:
Has anyone attempted to write a tool that allows copying of galaxy datasets locally, i.e. without downloading via http. We have scenario where there are some additional tools available only via command line and it would be very useful to be able to directly copy galaxy datasets from the datasets folder.
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
 https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
 http://galaxyproject.org/search/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
Reply | Threaded
Open this post in threaded view
|

Re: Tool to download galaxy datasets locally

Nicola Soranzo-2
Dear Evan,
you need to setup an environment variable for the tool to specify the
directory where users are expected to save their files. One way to do
that is:

1) create a directory called EXPORT_DIR_PREFIX in your tool dependencies
directory (which by default is `database/dependencies/` )
2) in this directory create a file called env.sh with a content similar
to the following template:

EXPORT_DIR_PREFIX=/PATH/WHERE/TO/SAVE/FILES; export EXPORT_DIR_PREFIX

3) create a symlink called `default` inside the EXPORT_DIR_PREFIX
directory pointing to the same directory, e.g.:

$ cd TOOL_DEPENDENCIES_DIR/EXPORT_DIR_PREFIX/
$ ln -s . default

Alternatively, you could define the EXPORT_DIR_PREFIX environment
variable in your config/job_conf.xml for the destination where the
export_to_cluster tool is going to run (for an example, see
`_JAVA_OPTIONS` in config/job_conf.xml.sample_advanced ), but this would
define the environment variable also for all the other tools that use
that destination (normally not a problem, but worth to mention).

Cheers,
Nicola

On 05/05/18 15:04, Evan Clark wrote:

> Have you ever seen this error with export to cluster?
>
> export_to_cluster.py: error: Directory prefix cannot be empty
>
> Nicola Soranzo wrote:
>>
>> Hi Evan,
>> you can have a look at
>> https://toolshed.g2.bx.psu.edu/view/earlhaminst/export_to_cluster/
>> <https://toolshed.g2.bx.psu.edu/view/earlhaminst/export_to_cluster/b97775e9fe06>
>>
>> Hope that helps!
>>
>> Cheers,
>> Nicola
>>
>> On 27/03/18 16:03, evan clark wrote:
>>>
>>> Has anyone attempted to write a tool that allows copying of galaxy
>>> datasets locally, i.e. without downloading via http. We have scenario
>>> where there are some additional tools available only via command line
>>> and it would be very useful to be able to directly copy galaxy
>>> datasets from the datasets folder.
>>> ___________________________________________________________
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>> https://lists.galaxyproject.org/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>> http://galaxyproject.org/search/
>>

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
Reply | Threaded
Open this post in threaded view
|

Re: Tool to download galaxy datasets locally

evan clark
Good afternoon,

Thank you for the information, due to the way our cluster storage is setup I just ended up editing the script to not use that variable as a check and adding a line to the tool xml that tells users to input only a certain directory in their home directory otherwise it will not work.

Nicola Soranzo wrote:

Dear Evan,
you need to setup an environment variable for the tool to specify the
directory where users are expected to save their files. One way to do
that is:

1) create a directory called EXPORT_DIR_PREFIX in your tool
dependencies directory (which by default is `database/dependencies/` )
2) in this directory create a file called env.sh with a content
similar to the following template:

EXPORT_DIR_PREFIX=/PATH/WHERE/TO/SAVE/FILES; export EXPORT_DIR_PREFIX

3) create a symlink called `default` inside the EXPORT_DIR_PREFIX
directory pointing to the same directory, e.g.:

$ cd TOOL_DEPENDENCIES_DIR/EXPORT_DIR_PREFIX/
$ ln -s . default

Alternatively, you could define the EXPORT_DIR_PREFIX environment
variable in your config/job_conf.xml for the destination where the
export_to_cluster tool is going to run (for an example, see
`_JAVA_OPTIONS` in config/job_conf.xml.sample_advanced ), but this
would define the environment variable also for all the other tools
that use that destination (normally not a problem, but worth to mention).

Cheers,
Nicola

On 05/05/18 15:04, Evan Clark wrote:

Have you ever seen this error with export to cluster?

export_to_cluster.py: error: Directory prefix cannot be empty

Nicola Soranzo wrote:


Hi Evan,
you can have a look at
https://toolshed.g2.bx.psu.edu/view/earlhaminst/export_to_cluster/
<https://toolshed.g2.bx.psu.edu/view/earlhaminst/export_to_cluster/b97775e9fe06>


Hope that helps!

Cheers,
Nicola

On 27/03/18 16:03, evan clark wrote:


Has anyone attempted to write a tool that allows copying of galaxy
datasets locally, i.e. without downloading via http. We have scenario
where there are some additional tools available only via command line
and it would be very useful to be able to directly copy galaxy
datasets from the datasets folder.
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/





___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/