Bio-Bigfile fixes

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Bio-Bigfile fixes

Fields, Christopher J
All, 

I ran into an API change with bigwig file support in the latest kent utils (off their git repo); I committed a fix as a branch to GBrowse_Adaptors:


One problem: this will obviously complicate compilation of older versions of the kent lib still floating about.  Should we run a version check of the kent API to make sure this makes sense?  Or is this even possible with the current source tree?

chris

------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: Bio-Bigfile fixes

Timothy Parnell
Hi Chris,

Thanks for discovering this.

Do you know if the jksrc.zip located at the UCSC website includes this change too? This is the link that that most documentation points to, rather than the GitHub repo.

I have always been slightly annoyed that the jksrc.zip download is never labeled with a version number or date. I have had issues in the past with different versions of jksrc that yielded different behaviors with bigWig files….

I am not aware of a version number in their source code. I suppose a fancy grep could figure out which code library version a user is installing, but that will inevitably break.

You may want, at the very least, to update the README with directions to update the kent library prior to installing the module. Hopefully with errors, users will take the time to read the README.

Tim

On Jan 7, 2014, at 2:32 PM, "Fields, Christopher J" <[hidden email]<mailto:[hidden email]>> wrote:

All,

I ran into an API change with bigwig file support in the latest kent utils (off their git repo); I committed a fix as a branch to GBrowse_Adaptors:

https://github.com/GMOD/GBrowse-Adaptors/tree/topic/kent_jan2014

One problem: this will obviously complicate compilation of older versions of the kent lib still floating about.  Should we run a version check of the kent API to make sure this makes sense?  Or is this even possible with the current source tree?

chris
------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse


------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: Bio-Bigfile fixes

Fields, Christopher J
On Jan 7, 2014, at 5:06 PM, Timothy Parnell <[hidden email]> wrote:

> Hi Chris,
>
> Thanks for discovering this.
>
> Do you know if the jksrc.zip located at the UCSC website includes this change too? This is the link that that most documentation points to, rather than the GitHub repo.

No, but I do expect this down the way; here’s the commit:

    http://genome-source.cse.ucsc.edu/gitweb/?p=kent.git;a=commit;h=e947f44bed3e61a500f59423d3c560eff9bcf270

Not in the ‘beta' branch yet, but a merge there should be imminent (it would likely appear in the next release and in jksrc.zip sometime after).

> I have always been slightly annoyed that the jksrc.zip download is never labeled with a version number or date. I have had issues in the past with different versions of jksrc that yielded different behaviors with bigWig files….
>
> I am not aware of a version number in their source code. I suppose a fancy grep could figure out which code library version a user is installing, but that will inevitably break.

Looks like there is a version in a header and in a build script:

   src/hg/inc/versionInfo.h

It appears that (in their beta and master branch and in the latest jksrc.zip) this is currently defined as 293 (looks as if they are working on preview releases of 294).  My feeling is the best approach might be to make this as a simple patch until we know this is actually incorporated.

> You may want, at the very least, to update the README with directions to update the kent library prior to installing the module. Hopefully with errors, users will take the time to read the README.
>
> Tim

We may want to add a section to the README mentioning this and simply point out the patch for the time being, but it appears there is a way to track the version when this does come down.

chris

> On Jan 7, 2014, at 2:32 PM, "Fields, Christopher J" <[hidden email]<mailto:[hidden email]>> wrote:
>
> All,
>
> I ran into an API change with bigwig file support in the latest kent utils (off their git repo); I committed a fix as a branch to GBrowse_Adaptors:
>
> https://github.com/GMOD/GBrowse-Adaptors/tree/topic/kent_jan2014
>
> One problem: this will obviously complicate compilation of older versions of the kent lib still floating about.  Should we run a version check of the kent API to make sure this makes sense?  Or is this even possible with the current source tree?
>
> chris
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most IT
> organizations don't have a clear picture of how application performance
> affects their revenue. With AppDynamics, you get 100% visibility into your
> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk_______________________________________________
> Gmod-gbrowse mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
>


------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: Bio-Bigfile fixes

Keiran Raine
Hi guys,

Looking at the whole commit it looks like bamToGbrowse.pl could be updated to use significantly less memory in the wig to bigwig step using the bigWigCat c program after building the bigwig files on a per chromosome basis rather than in one.

Probably a good idea to keep a close eye on this

Regards,
Keiran

> On 8 Jan 2014, at 06:12, "Fields, Christopher J" <[hidden email]> wrote:
>
>> On Jan 7, 2014, at 5:06 PM, Timothy Parnell <[hidden email]> wrote:
>>
>> Hi Chris,
>>
>> Thanks for discovering this.
>>
>> Do you know if the jksrc.zip located at the UCSC website includes this change too? This is the link that that most documentation points to, rather than the GitHub repo.
>
> No, but I do expect this down the way; here’s the commit:
>
>    http://genome-source.cse.ucsc.edu/gitweb/?p=kent.git;a=commit;h=e947f44bed3e61a500f59423d3c560eff9bcf270
>
> Not in the ‘beta' branch yet, but a merge there should be imminent (it would likely appear in the next release and in jksrc.zip sometime after).
>
>> I have always been slightly annoyed that the jksrc.zip download is never labeled with a version number or date. I have had issues in the past with different versions of jksrc that yielded different behaviors with bigWig files….
>>
>> I am not aware of a version number in their source code. I suppose a fancy grep could figure out which code library version a user is installing, but that will inevitably break.
>
> Looks like there is a version in a header and in a build script:
>
>   src/hg/inc/versionInfo.h
>
> It appears that (in their beta and master branch and in the latest jksrc.zip) this is currently defined as 293 (looks as if they are working on preview releases of 294).  My feeling is the best approach might be to make this as a simple patch until we know this is actually incorporated.
>
>> You may want, at the very least, to update the README with directions to update the kent library prior to installing the module. Hopefully with errors, users will take the time to read the README.
>>
>> Tim
>
> We may want to add a section to the README mentioning this and simply point out the patch for the time being, but it appears there is a way to track the version when this does come down.
>
> chris
>
>> On Jan 7, 2014, at 2:32 PM, "Fields, Christopher J" <[hidden email]<mailto:[hidden email]>> wrote:
>>
>> All,
>>
>> I ran into an API change with bigwig file support in the latest kent utils (off their git repo); I committed a fix as a branch to GBrowse_Adaptors:
>>
>> https://github.com/GMOD/GBrowse-Adaptors/tree/topic/kent_jan2014
>>
>> One problem: this will obviously complicate compilation of older versions of the kent lib still floating about.  Should we run a version check of the kent API to make sure this makes sense?  Or is this even possible with the current source tree?
>>
>> chris
>> ------------------------------------------------------------------------------
>> Rapidly troubleshoot problems before they affect your business. Most IT
>> organizations don't have a clear picture of how application performance
>> affects their revenue. With AppDynamics, you get 100% visibility into your
>> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk_______________________________________________
>> Gmod-gbrowse mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
>
>
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most IT
> organizations don't have a clear picture of how application performance
> affects their revenue. With AppDynamics, you get 100% visibility into your
> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
> _______________________________________________
> Gmod-gbrowse mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse