User file access without uploading?

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

User file access without uploading?

Michael Siebauer
  Hi,

is there a way/module or data Library that allows users to access there
files WITHOUT the need of uploading them first? Cause our Galaxy
instance is running within the intranet, all files could be accessed
straight via NFS.

Thx, Michael ;-)
_______________________________________________
galaxy-dev mailing list
[hidden email]
http://lists.bx.psu.edu/listinfo/galaxy-dev
Reply | Threaded
Open this post in threaded view
|

Re: User file access without uploading?

Dennis Gascoigne-2
When you import to a library (uploading from preconfigured server
directory) there is an option "No" to not copy, and instead use in
place.

We played with this for a while then decided against it. moving the
source fileor editing it etc. can cause havoc.

Cheers
Dennis

On Wed, Jun 30, 2010 at 8:01 PM, Michael Siebauer
<[hidden email]> wrote:

>  Hi,
>
> is there a way/module or data Library that allows users to access there
> files WITHOUT the need of uploading them first? Cause our Galaxy instance is
> running within the intranet, all files could be accessed straight via NFS.
>
> Thx, Michael ;-)
> _______________________________________________
> galaxy-dev mailing list
> [hidden email]
> http://lists.bx.psu.edu/listinfo/galaxy-dev
>

_______________________________________________
galaxy-dev mailing list
[hidden email]
http://lists.bx.psu.edu/listinfo/galaxy-dev
Reply | Threaded
Open this post in threaded view
|

Re: User file access without uploading?

Michael Siebauer
  Hmm but there is no way, for a user to upload (without copying) his
files. Only the Admin User can create and fill data libraries, right?

Cheers, Michael

Am 30.06.2010 12:17, schrieb Dennis Gascoigne:

> When you import to a library (uploading from preconfigured server
> directory) there is an option "No" to not copy, and instead use in
> place.
>
> We played with this for a while then decided against it. moving the
> source fileor editing it etc. can cause havoc.
>
> Cheers
> Dennis
>
> On Wed, Jun 30, 2010 at 8:01 PM, Michael Siebauer
> <[hidden email]>  wrote:
>>   Hi,
>>
>> is there a way/module or data Library that allows users to access there
>> files WITHOUT the need of uploading them first? Cause our Galaxy instance is
>> running within the intranet, all files could be accessed straight via NFS.
>>
>> Thx, Michael ;-)
>> _______________________________________________
>> galaxy-dev mailing list
>> [hidden email]
>> http://lists.bx.psu.edu/listinfo/galaxy-dev
>>
>

_______________________________________________
galaxy-dev mailing list
[hidden email]
http://lists.bx.psu.edu/listinfo/galaxy-dev
Reply | Threaded
Open this post in threaded view
|

Re: User file access without uploading?

Greg Von Kuster
Non-admin users can upload directories of files as well as long as the admin has set the following config in universe_wsgi.ini:

# The following can be configured to allow non-admin users to upload a directory of files.  The
# configured directory must contain sub-directories named the same as the non-admin user's Galaxy
# login ( email ).  The non-admin user is restricted to uploading files or sub-directories of files
# contained in their directory.
user_library_import_dir = <some directory on the Galaxy instance's file system>
allow_library_path_paste = True 


On Jun 30, 2010, at 8:35 AM, Michael Siebauer wrote:

Hmm but there is no way, for a user to upload (without copying) his files. Only the Admin User can create and fill data libraries, right?

Cheers, Michael

Am 30.06.2010 12:17, schrieb Dennis Gascoigne:
When you import to a library (uploading from preconfigured server
directory) there is an option "No" to not copy, and instead use in
place.

We played with this for a while then decided against it. moving the
source fileor editing it etc. can cause havoc.

Cheers
Dennis

On Wed, Jun 30, 2010 at 8:01 PM, Michael Siebauer
<[hidden email]>  wrote:
 Hi,

is there a way/module or data Library that allows users to access there
files WITHOUT the need of uploading them first? Cause our Galaxy instance is
running within the intranet, all files could be accessed straight via NFS.

Thx, Michael ;-)
_______________________________________________
galaxy-dev mailing list
[hidden email]
http://lists.bx.psu.edu/listinfo/galaxy-dev



_______________________________________________
galaxy-dev mailing list
[hidden email]
http://lists.bx.psu.edu/listinfo/galaxy-dev

Greg Von Kuster
Galaxy Development Team




_______________________________________________
galaxy-dev mailing list
[hidden email]
http://lists.bx.psu.edu/listinfo/galaxy-dev
Reply | Threaded
Open this post in threaded view
|

Re: User file access without uploading?

Greg Von Kuster
I forgot to mention that the user must also have a role that is associated with the LIBRARY_ADD permission on the data library or folder to which they want to upload.  For information about all of the data library features, see our wiki at http://bitbucket.org/galaxy/galaxy-central/wiki/DataLibraries/Libraries.

On Jun 30, 2010, at 9:01 AM, Greg Von Kuster wrote:

Non-admin users can upload directories of files as well as long as the admin has set the following config in universe_wsgi.ini:

# The following can be configured to allow non-admin users to upload a directory of files.  The
# configured directory must contain sub-directories named the same as the non-admin user's Galaxy
# login ( email ).  The non-admin user is restricted to uploading files or sub-directories of files
# contained in their directory.
user_library_import_dir = <some directory on the Galaxy instance's file system>
allow_library_path_paste = True 


On Jun 30, 2010, at 8:35 AM, Michael Siebauer wrote:

Hmm but there is no way, for a user to upload (without copying) his files. Only the Admin User can create and fill data libraries, right?

Cheers, Michael

Am 30.06.2010 12:17, schrieb Dennis Gascoigne:
When you import to a library (uploading from preconfigured server
directory) there is an option "No" to not copy, and instead use in
place.

We played with this for a while then decided against it. moving the
source fileor editing it etc. can cause havoc.

Cheers
Dennis

On Wed, Jun 30, 2010 at 8:01 PM, Michael Siebauer
<[hidden email]>  wrote:
 Hi,

is there a way/module or data Library that allows users to access there
files WITHOUT the need of uploading them first? Cause our Galaxy instance is
running within the intranet, all files could be accessed straight via NFS.

Thx, Michael ;-)
_______________________________________________
galaxy-dev mailing list
[hidden email]
http://lists.bx.psu.edu/listinfo/galaxy-dev



_______________________________________________
galaxy-dev mailing list
[hidden email]
http://lists.bx.psu.edu/listinfo/galaxy-dev

Greg Von Kuster
Galaxy Development Team



_______________________________________________
galaxy-dev mailing list
[hidden email]
http://lists.bx.psu.edu/listinfo/galaxy-dev

Greg Von Kuster
Galaxy Development Team




_______________________________________________
galaxy-dev mailing list
[hidden email]
http://lists.bx.psu.edu/listinfo/galaxy-dev
Reply | Threaded
Open this post in threaded view
|

Re: User file access without uploading?

Dennis Gascoigne-2
In reply to this post by Greg Von Kuster
If you want another option, I wrote a quick tool wrapper and bash
script. It's a total hack but it works.
* create two files server_copy.sh and server_copy.xml in
<<GALAXYDIR>>/tools/data_source with the following content.
* Add the following line somewhere in your tool_conf <tool
file="data_source/server_copy.xml"/>

Cheers
Dennis


##################<<bash script server_copy.sh>>
######################(this is ridiculously simple and does NO error
check or validation - I have pretty good users)
#!/bin/sh

cp $*






##################<<XML script>>######################
<?xml version="1.0"?>
<tool name="Server Upload" id="server_upload">
      <!-- This tool is used when the user needs to access files
stored on galaxy's server without routing through the client
machine-->
      <description></description>
      <command interpreter="bash">server_copy.sh $input $output </command>
      <inputs>
            <param name="input" type="text" area="true" size="2x200"
label="Full path to source file" help="The file must be specified as
the full path - i.e. /home/user.name/mydata/foo.bar"/>
            <param name="out_format" type="select" label="Output data type">
                  <option value="fasta">FASTA</option>
                  <option value="fastqillumina">FASTQ Illumina (or
other qual=64)</option>
                  <option value="fastqsanger">FASTQ Sanger (or other
qual=33)</option>
                  <option value="sam">SAM</option>
                  <option value="bed">BED</option>
                  <option value="text">TXT</option>
                  <option value="interval">Interval</option>
            </param>
            <param format="txt" name="fname" type="text" size="200"
label="Enter a name for your file."/>
      </inputs>
      <outputs>
            <data format="text" name="output" size="200"
label="Moved: ${fname.value} " >
                  <change_format>
                        <when input="out_format" value="fasta" format="fasta" />
                        <when input="out_format" value="fastqillumina"
format="fastqillumina" />
                        <when input="out_format" value="fastqsanger"
format="fastqsanger" />
                        <when input="out_format" value="sam" format="sam" />
                        <when input="out_format" value="interval"
format="interval" />
                        <when input="out_format" value="bed" format="bed" />
                        <when input="out_format" value="text" format="text" />
                  </change_format>
            </data>
      </outputs>
</tool>


On Wed, Jun 30, 2010 at 11:01 PM, Greg Von Kuster <[hidden email]> wrote:

> Non-admin users can upload directories of files as well as long as the admin
> has set the following config in universe_wsgi.ini:
> # The following can be configured to allow non-admin users to upload a
> directory of files.  The
> # configured directory must contain sub-directories named the same as the
> non-admin user's Galaxy
> # login ( email ).  The non-admin user is restricted to uploading files or
> sub-directories of files
> # contained in their directory.
> user_library_import_dir = <some directory on the Galaxy instance's file
> system>
> allow_library_path_paste = True
>
> On Jun 30, 2010, at 8:35 AM, Michael Siebauer wrote:
>
> Hmm but there is no way, for a user to upload (without copying) his files.
> Only the Admin User can create and fill data libraries, right?
>
> Cheers, Michael
>
> Am 30.06.2010 12:17, schrieb Dennis Gascoigne:
>
> When you import to a library (uploading from preconfigured server
>
> directory) there is an option "No" to not copy, and instead use in
>
> place.
>
> We played with this for a while then decided against it. moving the
>
> source fileor editing it etc. can cause havoc.
>
> Cheers
>
> Dennis
>
> On Wed, Jun 30, 2010 at 8:01 PM, Michael Siebauer
>
> <[hidden email]>  wrote:
>
>  Hi,
>
> is there a way/module or data Library that allows users to access there
>
> files WITHOUT the need of uploading them first? Cause our Galaxy instance is
>
> running within the intranet, all files could be accessed straight via NFS.
>
> Thx, Michael ;-)
>
> _______________________________________________
>
> galaxy-dev mailing list
>
> [hidden email]
>
> http://lists.bx.psu.edu/listinfo/galaxy-dev
>
>
>
> _______________________________________________
> galaxy-dev mailing list
> [hidden email]
> http://lists.bx.psu.edu/listinfo/galaxy-dev
>
> Greg Von Kuster
> Galaxy Development Team
> [hidden email]
>
>
>
> _______________________________________________
> galaxy-dev mailing list
> [hidden email]
> http://lists.bx.psu.edu/listinfo/galaxy-dev
>
>

_______________________________________________
galaxy-dev mailing list
[hidden email]
http://lists.bx.psu.edu/listinfo/galaxy-dev
Reply | Threaded
Open this post in threaded view
|

Re: User file access without uploading?

Assaf Gordon-2
Dennis,

Please note that your shell script is slightly dangerous and prone to malicious attacks.
Not too bad if all your users anyhow have local accounts on your server, but generally speaking, it's risky because it would allow users without local access to get information on your system.


The simplest example is a user asking to copy a sensitive file, like "/etc/passwd".
A denial-of-service would be to ask to copy special files like "/etc/urandom" - will fill up your disk.

But it gets  more dangerous because you don't validate the options at all,
so if I use this string as the input file:
 "--target-directory /tmp /etc/passwd"

You'll pass it directly to the "cp" program, and "cp" will treat all arguments as input files and copy them to "/tmp" - effectively allowing to write to ANY location on your disk that is writable by your galaxy user.

Not to mention that with a little bit of probing, one can find your "universe_wsgi.ini" and see secrets/passwords of your galaxy, and the database password.

I would recommend at the very minimum, to change the shell script as follows (this would disable arbitrary writing):
===
#!/bin/sh

INPUT="$1"
OUTPUT="$2"

[ -f "$INPUT" ] || { echo "Error: input file '$INPUT' is not a valid file" >&2 ; exit 1 ; }
cp -- "$INPUT" "$OUTPUT" || exit 1
===

To be even more careful, I would check the absolute path of the input file against a list of locations you allow:
===
#!/bin/sh

INPUT="$1"
OUTPUT="$2"

#the file must exist (-e)
ABS=$(readlink -en -- "$INPUT")
[ -f "$ABS" ] || { echo "Error: input file '$INPUT' is not a valid file" >&2 ; exit 1 ; }
 
# Extract the first 6 characters, make sure they are "/home/"
PREFIX=${ABS:0:6}
[ "x$PREFIX" = "x/home/" ] || { echo "Invalid input file path ($ABS)" >&2 ; exit 1 ; }
 
cp -- "$INPUT" "$OUTPUT" || exit 1
===

But even that would not stop users from getting other users' files (if they are readable by your galaxy user).

If you must allow users to get files directly from the server,
I would recommend setting up a public place (just one directory) to which users can copy the files, and your script will allow importing files only from that folder.

-gordon


Dennis Gascoigne wrote, On 06/30/2010 05:53 PM:

> If you want another option, I wrote a quick tool wrapper and bash
> script. It's a total hack but it works.
> * create two files server_copy.sh and server_copy.xml in
> <<GALAXYDIR>>/tools/data_source with the following content.
> * Add the following line somewhere in your tool_conf <tool
> file="data_source/server_copy.xml"/>
>
> Cheers
> Dennis
>
>
> ##################<<bash script server_copy.sh>>
> ######################(this is ridiculously simple and does NO error
> check or validation - I have pretty good users)
> #!/bin/sh
>
> cp $*
>
>
>
>
>
>
> ##################<<XML script>>######################
> <?xml version="1.0"?>
> <tool name="Server Upload" id="server_upload">
>       <!-- This tool is used when the user needs to access files
> stored on galaxy's server without routing through the client
> machine-->
>       <description></description>
>       <command interpreter="bash">server_copy.sh $input $output </command>
>       <inputs>
>             <param name="input" type="text" area="true" size="2x200"
> label="Full path to source file" help="The file must be specified as
> the full path - i.e. /home/user.name/mydata/foo.bar"/>
>             <param name="out_format" type="select" label="Output data type">
>                   <option value="fasta">FASTA</option>
>                   <option value="fastqillumina">FASTQ Illumina (or
> other qual=64)</option>
>                   <option value="fastqsanger">FASTQ Sanger (or other
> qual=33)</option>
>                   <option value="sam">SAM</option>
>                   <option value="bed">BED</option>
>                   <option value="text">TXT</option>
>                   <option value="interval">Interval</option>
>             </param>
>             <param format="txt" name="fname" type="text" size="200"
> label="Enter a name for your file."/>
>       </inputs>
>       <outputs>
>             <data format="text" name="output" size="200"
> label="Moved: ${fname.value} " >
>                   <change_format>
>                         <when input="out_format" value="fasta" format="fasta" />
>                         <when input="out_format" value="fastqillumina"
> format="fastqillumina" />
>                         <when input="out_format" value="fastqsanger"
> format="fastqsanger" />
>                         <when input="out_format" value="sam" format="sam" />
>                         <when input="out_format" value="interval"
> format="interval" />
>                         <when input="out_format" value="bed" format="bed" />
>                         <when input="out_format" value="text" format="text" />
>                   </change_format>
>             </data>
>       </outputs>
> </tool>
>
>
_______________________________________________
galaxy-dev mailing list
[hidden email]
http://lists.bx.psu.edu/listinfo/galaxy-dev
Reply | Threaded
Open this post in threaded view
|

Re: User file access without uploading?

Assaf Gordon-2
I forgot the add that in the XML file,
One should also quote the two parameters, to make sure the $input variable always counts a single parameter, even if it contains spaces or tabs:

change from:
 <command interpreter="bash">server_copy.sh $input $output </command>

to
 <command interpreter="bash">server_copy.sh '$input' '$output' </command>


Assaf Gordon wrote, On 06/30/2010 06:47 PM:

> Dennis,
>
> Please note that your shell script is slightly dangerous and prone to malicious attacks.
> Not too bad if all your users anyhow have local accounts on your server, but generally speaking, it's risky because it would allow users without local access to get information on your system.
>
>
> The simplest example is a user asking to copy a sensitive file, like "/etc/passwd".
> A denial-of-service would be to ask to copy special files like "/etc/urandom" - will fill up your disk.
>
> But it gets  more dangerous because you don't validate the options at all,
> so if I use this string as the input file:
>  "--target-directory /tmp /etc/passwd"
>
> You'll pass it directly to the "cp" program, and "cp" will treat all arguments as input files and copy them to "/tmp" - effectively allowing to write to ANY location on your disk that is writable by your galaxy user.
>
> Not to mention that with a little bit of probing, one can find your "universe_wsgi.ini" and see secrets/passwords of your galaxy, and the database password.
>
> I would recommend at the very minimum, to change the shell script as follows (this would disable arbitrary writing):
> ===
> #!/bin/sh
>
> INPUT="$1"
> OUTPUT="$2"
>
> [ -f "$INPUT" ] || { echo "Error: input file '$INPUT' is not a valid file" >&2 ; exit 1 ; }
> cp -- "$INPUT" "$OUTPUT" || exit 1
> ===
>
> To be even more careful, I would check the absolute path of the input file against a list of locations you allow:
> ===
> #!/bin/sh
>
> INPUT="$1"
> OUTPUT="$2"
>
> #the file must exist (-e)
> ABS=$(readlink -en -- "$INPUT")
> [ -f "$ABS" ] || { echo "Error: input file '$INPUT' is not a valid file" >&2 ; exit 1 ; }
>  
> # Extract the first 6 characters, make sure they are "/home/"
> PREFIX=${ABS:0:6}
> [ "x$PREFIX" = "x/home/" ] || { echo "Invalid input file path ($ABS)" >&2 ; exit 1 ; }
>  
> cp -- "$INPUT" "$OUTPUT" || exit 1
> ===
>
> But even that would not stop users from getting other users' files (if they are readable by your galaxy user).
>
> If you must allow users to get files directly from the server,
> I would recommend setting up a public place (just one directory) to which users can copy the files, and your script will allow importing files only from that folder.
>
> -gordon
>
>
> Dennis Gascoigne wrote, On 06/30/2010 05:53 PM:
>> If you want another option, I wrote a quick tool wrapper and bash
>> script. It's a total hack but it works.
>> * create two files server_copy.sh and server_copy.xml in
>> <<GALAXYDIR>>/tools/data_source with the following content.
>> * Add the following line somewhere in your tool_conf <tool
>> file="data_source/server_copy.xml"/>
>>
>> Cheers
>> Dennis
>>
>>
>> ##################<<bash script server_copy.sh>>
>> ######################(this is ridiculously simple and does NO error
>> check or validation - I have pretty good users)
>> #!/bin/sh
>>
>> cp $*
>>
>>
>>
>>
>>
>>
>> ##################<<XML script>>######################
>> <?xml version="1.0"?>
>> <tool name="Server Upload" id="server_upload">
>>       <!-- This tool is used when the user needs to access files
>> stored on galaxy's server without routing through the client
>> machine-->
>>       <description></description>
>>       <command interpreter="bash">server_copy.sh $input $output </command>
>>       <inputs>
>>             <param name="input" type="text" area="true" size="2x200"
>> label="Full path to source file" help="The file must be specified as
>> the full path - i.e. /home/user.name/mydata/foo.bar"/>
>>             <param name="out_format" type="select" label="Output data type">
>>                   <option value="fasta">FASTA</option>
>>                   <option value="fastqillumina">FASTQ Illumina (or
>> other qual=64)</option>
>>                   <option value="fastqsanger">FASTQ Sanger (or other
>> qual=33)</option>
>>                   <option value="sam">SAM</option>
>>                   <option value="bed">BED</option>
>>                   <option value="text">TXT</option>
>>                   <option value="interval">Interval</option>
>>             </param>
>>             <param format="txt" name="fname" type="text" size="200"
>> label="Enter a name for your file."/>
>>       </inputs>
>>       <outputs>
>>             <data format="text" name="output" size="200"
>> label="Moved: ${fname.value} " >
>>                   <change_format>
>>                         <when input="out_format" value="fasta" format="fasta" />
>>                         <when input="out_format" value="fastqillumina"
>> format="fastqillumina" />
>>                         <when input="out_format" value="fastqsanger"
>> format="fastqsanger" />
>>                         <when input="out_format" value="sam" format="sam" />
>>                         <when input="out_format" value="interval"
>> format="interval" />
>>                         <when input="out_format" value="bed" format="bed" />
>>                         <when input="out_format" value="text" format="text" />
>>                   </change_format>
>>             </data>
>>       </outputs>
>> </tool>
>>
>>
> _______________________________________________
> galaxy-dev mailing list
> [hidden email]
> http://lists.bx.psu.edu/listinfo/galaxy-dev

_______________________________________________
galaxy-dev mailing list
[hidden email]
http://lists.bx.psu.edu/listinfo/galaxy-dev
Reply | Threaded
Open this post in threaded view
|

Re: User file access without uploading?

Dennis Gascoigne-2
Assaf;

Yes, I should have pointed that out. Thankyou for doing so for me.

Ours is a small group of users on a tightly locked down system with no
public access, galaxy running in a restricted user account and where
the users are both responsible and more importantly, i know where they
live so I can hunt them down if they do anything bad!

Dennis

On Thu, Jul 1, 2010 at 8:51 AM, Assaf Gordon <[hidden email]> wrote:

> I forgot the add that in the XML file,
> One should also quote the two parameters, to make sure the $input variable always counts a single parameter, even if it contains spaces or tabs:
>
> change from:
>  <command interpreter="bash">server_copy.sh $input $output </command>
>
> to
>  <command interpreter="bash">server_copy.sh '$input' '$output' </command>
>
>
> Assaf Gordon wrote, On 06/30/2010 06:47 PM:
>> Dennis,
>>
>> Please note that your shell script is slightly dangerous and prone to malicious attacks.
>> Not too bad if all your users anyhow have local accounts on your server, but generally speaking, it's risky because it would allow users without local access to get information on your system.
>>
>>
>> The simplest example is a user asking to copy a sensitive file, like "/etc/passwd".
>> A denial-of-service would be to ask to copy special files like "/etc/urandom" - will fill up your disk.
>>
>> But it gets  more dangerous because you don't validate the options at all,
>> so if I use this string as the input file:
>>  "--target-directory /tmp /etc/passwd"
>>
>> You'll pass it directly to the "cp" program, and "cp" will treat all arguments as input files and copy them to "/tmp" - effectively allowing to write to ANY location on your disk that is writable by your galaxy user.
>>
>> Not to mention that with a little bit of probing, one can find your "universe_wsgi.ini" and see secrets/passwords of your galaxy, and the database password.
>>
>> I would recommend at the very minimum, to change the shell script as follows (this would disable arbitrary writing):
>> ===
>> #!/bin/sh
>>
>> INPUT="$1"
>> OUTPUT="$2"
>>
>> [ -f "$INPUT" ] || { echo "Error: input file '$INPUT' is not a valid file" >&2 ; exit 1 ; }
>> cp -- "$INPUT" "$OUTPUT" || exit 1
>> ===
>>
>> To be even more careful, I would check the absolute path of the input file against a list of locations you allow:
>> ===
>> #!/bin/sh
>>
>> INPUT="$1"
>> OUTPUT="$2"
>>
>> #the file must exist (-e)
>> ABS=$(readlink -en -- "$INPUT")
>> [ -f "$ABS" ] || { echo "Error: input file '$INPUT' is not a valid file" >&2 ; exit 1 ; }
>>
>> # Extract the first 6 characters, make sure they are "/home/"
>> PREFIX=${ABS:0:6}
>> [ "x$PREFIX" = "x/home/" ] || { echo "Invalid input file path ($ABS)" >&2 ; exit 1 ; }
>>
>> cp -- "$INPUT" "$OUTPUT" || exit 1
>> ===
>>
>> But even that would not stop users from getting other users' files (if they are readable by your galaxy user).
>>
>> If you must allow users to get files directly from the server,
>> I would recommend setting up a public place (just one directory) to which users can copy the files, and your script will allow importing files only from that folder.
>>
>> -gordon
>>
>>
>> Dennis Gascoigne wrote, On 06/30/2010 05:53 PM:
>>> If you want another option, I wrote a quick tool wrapper and bash
>>> script. It's a total hack but it works.
>>> * create two files server_copy.sh and server_copy.xml in
>>> <<GALAXYDIR>>/tools/data_source with the following content.
>>> * Add the following line somewhere in your tool_conf <tool
>>> file="data_source/server_copy.xml"/>
>>>
>>> Cheers
>>> Dennis
>>>
>>>
>>> ##################<<bash script server_copy.sh>>
>>> ######################(this is ridiculously simple and does NO error
>>> check or validation - I have pretty good users)
>>> #!/bin/sh
>>>
>>> cp $*
>>>
>>>
>>>
>>>
>>>
>>>
>>> ##################<<XML script>>######################
>>> <?xml version="1.0"?>
>>> <tool name="Server Upload" id="server_upload">
>>>       <!-- This tool is used when the user needs to access files
>>> stored on galaxy's server without routing through the client
>>> machine-->
>>>       <description></description>
>>>       <command interpreter="bash">server_copy.sh $input $output </command>
>>>       <inputs>
>>>             <param name="input" type="text" area="true" size="2x200"
>>> label="Full path to source file" help="The file must be specified as
>>> the full path - i.e. /home/user.name/mydata/foo.bar"/>
>>>             <param name="out_format" type="select" label="Output data type">
>>>                   <option value="fasta">FASTA</option>
>>>                   <option value="fastqillumina">FASTQ Illumina (or
>>> other qual=64)</option>
>>>                   <option value="fastqsanger">FASTQ Sanger (or other
>>> qual=33)</option>
>>>                   <option value="sam">SAM</option>
>>>                   <option value="bed">BED</option>
>>>                   <option value="text">TXT</option>
>>>                   <option value="interval">Interval</option>
>>>             </param>
>>>             <param format="txt" name="fname" type="text" size="200"
>>> label="Enter a name for your file."/>
>>>       </inputs>
>>>       <outputs>
>>>             <data format="text" name="output" size="200"
>>> label="Moved: ${fname.value} " >
>>>                   <change_format>
>>>                         <when input="out_format" value="fasta" format="fasta" />
>>>                         <when input="out_format" value="fastqillumina"
>>> format="fastqillumina" />
>>>                         <when input="out_format" value="fastqsanger"
>>> format="fastqsanger" />
>>>                         <when input="out_format" value="sam" format="sam" />
>>>                         <when input="out_format" value="interval"
>>> format="interval" />
>>>                         <when input="out_format" value="bed" format="bed" />
>>>                         <when input="out_format" value="text" format="text" />
>>>                   </change_format>
>>>             </data>
>>>       </outputs>
>>> </tool>
>>>
>>>
>> _______________________________________________
>> galaxy-dev mailing list
>> [hidden email]
>> http://lists.bx.psu.edu/listinfo/galaxy-dev
>
>

_______________________________________________
galaxy-dev mailing list
[hidden email]
http://lists.bx.psu.edu/listinfo/galaxy-dev
Reply | Threaded
Open this post in threaded view
|

Re: User file access without uploading?

Michael Siebauer
  Thank you very much. That solves my problem :-)

Am 01.07.2010 03:10, schrieb Dennis Gascoigne:

> Assaf;
>
> Yes, I should have pointed that out. Thankyou for doing so for me.
>
> Ours is a small group of users on a tightly locked down system with no
> public access, galaxy running in a restricted user account and where
> the users are both responsible and more importantly, i know where they
> live so I can hunt them down if they do anything bad!
>
> Dennis
>
> On Thu, Jul 1, 2010 at 8:51 AM, Assaf Gordon<[hidden email]>  wrote:
>> I forgot the add that in the XML file,
>> One should also quote the two parameters, to make sure the $input variable always counts a single parameter, even if it contains spaces or tabs:
>>
>> change from:
>>   <command interpreter="bash">server_copy.sh $input $output</command>
>>
>> to
>>   <command interpreter="bash">server_copy.sh '$input' '$output'</command>
>>
>>
>> Assaf Gordon wrote, On 06/30/2010 06:47 PM:
>>> Dennis,
>>>
>>> Please note that your shell script is slightly dangerous and prone to malicious attacks.
>>> Not too bad if all your users anyhow have local accounts on your server, but generally speaking, it's risky because it would allow users without local access to get information on your system.
>>>
>>>
>>> The simplest example is a user asking to copy a sensitive file, like "/etc/passwd".
>>> A denial-of-service would be to ask to copy special files like "/etc/urandom" - will fill up your disk.
>>>
>>> But it gets  more dangerous because you don't validate the options at all,
>>> so if I use this string as the input file:
>>>   "--target-directory /tmp /etc/passwd"
>>>
>>> You'll pass it directly to the "cp" program, and "cp" will treat all arguments as input files and copy them to "/tmp" - effectively allowing to write to ANY location on your disk that is writable by your galaxy user.
>>>
>>> Not to mention that with a little bit of probing, one can find your "universe_wsgi.ini" and see secrets/passwords of your galaxy, and the database password.
>>>
>>> I would recommend at the very minimum, to change the shell script as follows (this would disable arbitrary writing):
>>> ===
>>> #!/bin/sh
>>>
>>> INPUT="$1"
>>> OUTPUT="$2"
>>>
>>> [ -f "$INPUT" ] || { echo "Error: input file '$INPUT' is not a valid file">&2 ; exit 1 ; }
>>> cp -- "$INPUT" "$OUTPUT" || exit 1
>>> ===
>>>
>>> To be even more careful, I would check the absolute path of the input file against a list of locations you allow:
>>> ===
>>> #!/bin/sh
>>>
>>> INPUT="$1"
>>> OUTPUT="$2"
>>>
>>> #the file must exist (-e)
>>> ABS=$(readlink -en -- "$INPUT")
>>> [ -f "$ABS" ] || { echo "Error: input file '$INPUT' is not a valid file">&2 ; exit 1 ; }
>>>
>>> # Extract the first 6 characters, make sure they are "/home/"
>>> PREFIX=${ABS:0:6}
>>> [ "x$PREFIX" = "x/home/" ] || { echo "Invalid input file path ($ABS)">&2 ; exit 1 ; }
>>>
>>> cp -- "$INPUT" "$OUTPUT" || exit 1
>>> ===
>>>
>>> But even that would not stop users from getting other users' files (if they are readable by your galaxy user).
>>>
>>> If you must allow users to get files directly from the server,
>>> I would recommend setting up a public place (just one directory) to which users can copy the files, and your script will allow importing files only from that folder.
>>>
>>> -gordon
>>>
>>>
>>> Dennis Gascoigne wrote, On 06/30/2010 05:53 PM:
>>>> If you want another option, I wrote a quick tool wrapper and bash
>>>> script. It's a total hack but it works.
>>>> * create two files server_copy.sh and server_copy.xml in
>>>> <<GALAXYDIR>>/tools/data_source with the following content.
>>>> * Add the following line somewhere in your tool_conf<tool
>>>> file="data_source/server_copy.xml"/>
>>>>
>>>> Cheers
>>>> Dennis
>>>>
>>>>
>>>> ##################<<bash script server_copy.sh>>
>>>> ######################(this is ridiculously simple and does NO error
>>>> check or validation - I have pretty good users)
>>>> #!/bin/sh
>>>>
>>>> cp $*
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> ##################<<XML script>>######################
>>>> <?xml version="1.0"?>
>>>> <tool name="Server Upload" id="server_upload">
>>>>        <!-- This tool is used when the user needs to access files
>>>> stored on galaxy's server without routing through the client
>>>> machine-->
>>>>        <description></description>
>>>>        <command interpreter="bash">server_copy.sh $input $output</command>
>>>>        <inputs>
>>>>              <param name="input" type="text" area="true" size="2x200"
>>>> label="Full path to source file" help="The file must be specified as
>>>> the full path - i.e. /home/user.name/mydata/foo.bar"/>
>>>>              <param name="out_format" type="select" label="Output data type">
>>>>                    <option value="fasta">FASTA</option>
>>>>                    <option value="fastqillumina">FASTQ Illumina (or
>>>> other qual=64)</option>
>>>>                    <option value="fastqsanger">FASTQ Sanger (or other
>>>> qual=33)</option>
>>>>                    <option value="sam">SAM</option>
>>>>                    <option value="bed">BED</option>
>>>>                    <option value="text">TXT</option>
>>>>                    <option value="interval">Interval</option>
>>>>              </param>
>>>>              <param format="txt" name="fname" type="text" size="200"
>>>> label="Enter a name for your file."/>
>>>>        </inputs>
>>>>        <outputs>
>>>>              <data format="text" name="output" size="200"
>>>> label="Moved: ${fname.value} ">
>>>>                    <change_format>
>>>>                          <when input="out_format" value="fasta" format="fasta" />
>>>>                          <when input="out_format" value="fastqillumina"
>>>> format="fastqillumina" />
>>>>                          <when input="out_format" value="fastqsanger"
>>>> format="fastqsanger" />
>>>>                          <when input="out_format" value="sam" format="sam" />
>>>>                          <when input="out_format" value="interval"
>>>> format="interval" />
>>>>                          <when input="out_format" value="bed" format="bed" />
>>>>                          <when input="out_format" value="text" format="text" />
>>>>                    </change_format>
>>>>              </data>
>>>>        </outputs>
>>>> </tool>
>>>>
>>>>
>>> _______________________________________________
>>> galaxy-dev mailing list
>>> [hidden email]
>>> http://lists.bx.psu.edu/listinfo/galaxy-dev
>>
> _______________________________________________
> galaxy-dev mailing list
> [hidden email]
> http://lists.bx.psu.edu/listinfo/galaxy-dev
>
>

_______________________________________________
galaxy-dev mailing list
[hidden email]
http://lists.bx.psu.edu/listinfo/galaxy-dev