Enable viewing of large HTML files

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Enable viewing of large HTML files

Peter Briggs
Dear Devs

We have a locally developed tool which takes in a collection of Fastq pairs and produces a number of outputs which includes a number of HTML files.

For relatively small numbers of input files all the HTML outputs can be viewed without issue in Galaxy. However we've observed that for large collections of Fastqs, the size of one of the HTML files can be very large (several megabytes) and by default attempting to view these in Galaxy results in the "This dataset is large and only the first megabyte is shown below" message, followed by the initial HTML markup rendered as text.

As clicking on the "show all" link renders the entire HTML file correctly, is there a way to force Galaxy to shortcut the truncation for HTML (or at least apply a larger limit for determining when files are too large to be shown all at once)?

We're currently using 17.05 in our production instance.

Thanks for advance for any help or suggestions,

Best wishes

Peter


--
Peter Briggs [hidden email]
Bioinformatics Core Facility University of Manchester
B.1083 Michael Smith Bldg Tel: (0161) 2751482

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
Reply | Threaded
Open this post in threaded view
|

Re: Enable viewing of large HTML files

Martin Čech
Hi Peter,

I believe the 1MB is hard coded in https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/managers/hdas.py#L222 and cannot be configured.

That said you can change it on your instance, I have also opened an issue to allow configuring this value. https://github.com/galaxyproject/galaxy/issues/4924

Martin

On Fri, Nov 3, 2017 at 8:00 AM Peter Briggs <[hidden email]> wrote:
Dear Devs

We have a locally developed tool which takes in a collection of Fastq pairs and produces a number of outputs which includes a number of HTML files.

For relatively small numbers of input files all the HTML outputs can be viewed without issue in Galaxy. However we've observed that for large collections of Fastqs, the size of one of the HTML files can be very large (several megabytes) and by default attempting to view these in Galaxy results in the "This dataset is large and only the first megabyte is shown below" message, followed by the initial HTML markup rendered as text.

As clicking on the "show all" link renders the entire HTML file correctly, is there a way to force Galaxy to shortcut the truncation for HTML (or at least apply a larger limit for determining when files are too large to be shown all at once)?

We're currently using 17.05 in our production instance.

Thanks for advance for any help or suggestions,

Best wishes

Peter


--
Peter Briggs [hidden email]
Bioinformatics Core Facility University of Manchester
B.1083 Michael Smith Bldg Tel: (0161) 2751482

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
Reply | Threaded
Open this post in threaded view
|

Re: Enable viewing of large HTML files

Peter Briggs
Hello Martin

Thanks for the quick reply and for the pointer to the appropriate code, that's great - I'll take a look at tweaking the limit in our local Galaxy if it continues to be a problem.

Best wishes

Peter

-- 
Peter Briggs [hidden email]
Bioinformatics Core Facility University of Manchester
B.1083 Michael Smith Bldg Tel: (0161) 2751482


From: Martin Čech [[hidden email]]
Sent: Friday, November 03, 2017 2:47 PM
To: Peter Briggs
Cc: [hidden email]
Subject: Re: [galaxy-dev] Enable viewing of large HTML files

Hi Peter,

I believe the 1MB is hard coded in https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/managers/hdas.py#L222 and cannot be configured.

That said you can change it on your instance, I have also opened an issue to allow configuring this value. https://github.com/galaxyproject/galaxy/issues/4924

Martin

On Fri, Nov 3, 2017 at 8:00 AM Peter Briggs <[hidden email]> wrote:
Dear Devs

We have a locally developed tool which takes in a collection of Fastq pairs and produces a number of outputs which includes a number of HTML files.

For relatively small numbers of input files all the HTML outputs can be viewed without issue in Galaxy. However we've observed that for large collections of Fastqs, the size of one of the HTML files can be very large (several megabytes) and by default attempting to view these in Galaxy results in the "This dataset is large and only the first megabyte is shown below" message, followed by the initial HTML markup rendered as text.

As clicking on the "show all" link renders the entire HTML file correctly, is there a way to force Galaxy to shortcut the truncation for HTML (or at least apply a larger limit for determining when files are too large to be shown all at once)?

We're currently using 17.05 in our production instance.

Thanks for advance for any help or suggestions,

Best wishes

Peter


--
Peter Briggs [hidden email]
Bioinformatics Core Facility University of Manchester
B.1083 Michael Smith Bldg Tel: (0161) 2751482

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/